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PREFACE 



Despite the development of a now vast body of knowledge known as 
modern control theory, and despite some spectacular applications of this 
theory to practical situations, it is quite clear that much of the theory has 
yet to find application, and many practical control problems have yet to find 
a theory which will successfully deal with them. No book of course can 
remedy the situation at this time. But the aim of this book is to construct 
one of many bridges that are still required for the student and practicing 
control engineer between the familiar classical control results and those of 
modern control theory. It attempts to do so by consistently adopting the 
viewpoints that 

1. many modern control results have interpretation in terms of clas- 
sical control results; 

2. many modern control results do have practical engineering signif- 
icance, as distinct from applied mathematical significance. 

As a consequence, linear systems are very heavily emphasized, and, 
indeed, the discussion of nonlinear systems is essentially restricted to two 
classes: systems which should be linear, but unhappily are not; and systems 
which are linear but for the intentional use of relays. Also as a consequence 
of this approach, discussion of some results deemed fundamental in the 
general theory of optimal control has been kept to the barest minimum, 
thereby allowing emphasis on those particular optimal control results having 
application to linear systems. It may therefore seem strange to present a 
book on optimal control which does not discuss the Pontryagin Maximum 
Principle, but it is nonetheless consistent with the general aims of the book. 

Although the selection of the material for the book has not been governed 
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by the idea of locating the optimal control theory of linear systems within 
the broader framework of optimal control theory per se, it has been governed 
by the aim of presenting results of linear optimal control theory interesting 
from an engineering point of view, consistent with the ability of students to 
follow the material. This has not meant restricting the choice of material 
presented to that covered in other books; indeed a good many of the ideas 
discussed have appeared only in technical papers. 

For the most part, continuous time systems are treated, and a good 
deal more of the discussion is on time-invariant than is on time-varying 
systems. Infinite-time optimization problems for time-varying systems involve 
concepts such as uniform complete controllability, which the authors con- 
sider to be in the nature of advanced rather than core material, and accord- 
ingly discussion of such material is kept to a minimum. For completeness, 
some mention is also made of discrete-time systems, but it seemed to us 
that any extended discussion of discrete-time systems would involve undue 
repetition. 

The text is aimed at the first or later year graduate student. The back- 
ground assumed of any reader is, first, an elementary control course, covering 
such notions as transfer functions, Nyquist plots, root locus, etc., second, 
an elementary introduction to the state-space description of linear systems 
and the dual notions of complete controllability and complete observability, 
and third, an elementary introduction to linear algebra. However, exposure 
to a prior or concurrent course in optimal control is not assumed. For 
students who have had a prior course, or are taking concurrently a course 
in the general theory of optimal control or a specific aspect of the discipline 
such as time-optimal systems, a course based on this book will still provide 
in-depth knowledge of an important area of optimal control. 

Besides an introductory chapter and a final chapter on computational 
aspects of optimal linear system design, the book contains three major parts. 
The first of these outlines the basic theory of the linear regulator, for time- 
invariant and time-varying systems, emphasizing the former. The actual 
derivation of the optimal control law is via the Hamilton-Jacobi equation 
which is introduced using the Principle of Optimality. The infinite-time 
problem is considered, with the introduction of exponential weighting in 
the performance index used for time-invariant design as a novel feature. The 
second major part of the book outlines the engineering properties of the 
regulator, and attempts to give the reader a feeling for the use of the optimal 
linear regulator theory as a design tool. Degree of stability, phase and gain 
margin, tolerance of time delay, effect of nonlinearities, introduction of 
relays, design to achieve prescribed closed-loop poles, various sensitivity 
problems, state estimation and design of practical controllers are all con- 
sidered. The third major part of the book discusses extensions to the servo- 
mechanism problem, to the situation where the derivative of the control 
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may be limited (leading to dynamic feedback controllers) or the control 
itself may be limited in amplitude (leading to feedback controllers con- 
taining relays), and to recent results on output, as distinct from state 
feedback. Material on discrete time systems and additional material on 
time-varying continuous systems is also presented. The final part of the 
book, consisting of one chapter only, discusses approaches to the solution 
of Riccati equations, including approximate solution procedures based on 
singular perturbation theory. Appendices summarizing matrix theory and 
linear system theory results relevant to the material of the book are also 
included. 

Readers who have been introduced to the regulator problem elsewhere 
may find section 3 of chapter 3 a convenient starting point, unless review 
of the earlier material is required. 

We would like to emphasize that the manuscript was compiled as a 
truly joint effort; it would be difficult to distinguish completely who wrote 
what section and whose ideas were involved at each point in the develop- 
ment of the material. Both of us were surprised at the fact that working 
together we could achieve far more than either of us working independently, 
and we are thankful for the personal enrichment to our lives from the 
experience of working together. 

In listing acknowledgments, our former teachers Robert Newcomb 
and Dragoslav Siljak come immediately to mind as do our graduate students 
Peter Moylan and Konrad Hitz. In knowing these people each of us has 
learned that a teacher-student relationship can be infinitely more worth- 
while than the usual connotation of the words implies. We appreciate the 
direct help given by our students as we also appreciate the work done by 
Brian Thomas in drafting the various diagrams, and Sue Dorahy, Pam 
O’Sullivan and Lorraine Youngberry for typing the manuscript. We are 
happy to acknowledge the financial support of our research by the Australian 
Research Grants Committee. We mention our families — Dianne and 
Elizabeth Anderson, and Jan and Kevin Moore because they are a part of us. 
We mention all these various names along with our own in recognition of 

the world of people out of whom the world of ideas is born and for whom 
it exists. 



Brian D. O. Anderson 
John B. Moore 
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INTRODUCTION 




CHAPTER 



INTRODUCTION 



1.1 LINEAR OPTIMAL CONTROL 

The methods and techniques of what is now known as “classical control” 
will be familiar to most readers. In the main, the systems or plants that can 
be considered by using classical control ideas are linear, time invariant, and 
have a single input and a single output. The primary aim of the designer 
using classical control design methods is to stabilize a plant, whereas secon- 
dary aims may involve obtaining a certain transient response, bandwidth, 
steady state error, and so on. The designer’s methods are a combination of 
analytical ones (e.g., Laplace transform, Routh test), graphical ones (e.g., 
Nyquist plots, Nichols charts), and a good deal of empirically based knowl- 
edge (e.g., a certain class of compensator works satisfactorily for a certain 
class of plant). For high-order systems, multiple-input systems, or systems 
that do not possess the properties usually assumed in the classical control 
approach, the designer’s ingenuity is generally the limiting factor in achiev- 
ing a satisfactory design. 

Two of the main aims of modern, as opposed to classical, control are 
to deempiricize control system design and to present solutions to a much 
wider class of control problems than classical control can tackle. One of the 
major ways modern control sets out to achieve these aims is by providing an 
array of analytical design procedures that lessen the load of the design task 
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on the designer’s ingenuity and locate more of the load upon his mathematical 
ability and on the computational machines used in actually carrying out the 
design. 

Optimal control is one particular branch of modern control that sets 
out to provide analytical designs of a specially appealing type. The system 
which is the end result of an optimal design is not supposed merely to be 
stable, have a certain bandwidth, or satisfy any one of the desirable con- 
straints associated with classical control, but it is supposed to be the best 
possible system of a particular type— hence, the word optimal. If it is both 
optimal and possesses a number of the properties that classical control sug- 
gests are desirable, so much the better. 

Linear optimal control is a special sort of optimal control. The plant 
that is controlled is assumed linear, and the controller, the device which 
generates the optimal control, is constrained to be linear. That is, its output, 
the optimal control, is supposed to depend linearly on its input, which will 
consist of quantities derived from measurements on the plant. Of course, 
one may well ask: Why linear optimal control, as opposed simply to optimal 
control? A number of justifications may be advanced — for example, many 
engineering plants are linear prior to addition of a controller to them; a 
linear controller is simple to implement physically, and will frequently suffice. 

Other advantages of optimal control, when it is specifically linear, 
follow. 

1. Many optimal control problems do not have computable solutions, 
or they have solutions that may only be obtained with a great deal 
of computing effort. By contrast, nearly all linear optimal control 
problems have readily computable solutions. 

2. Linear optimal control results may be applied to nonlinear systems 
operating on a small signal basis. More precisely, suppose an optimal 
control has been developed for some nonlinear system with the 
assumption that this system will start in a certain initial state. Sup- 
pose, however, that the system starts in a slightly different initial 
state, for which there exists some other optimal control. Then a 
first approximation to the difference between the two optimal controls 
may normally be derived, if desired, by solving a linear optimal 
control problem (with all its attendant computational advantages). 
This holds independently of the criterion for optimality for the non- 
linear system. (Since this topic will not be discussed anywhere in this 
book, we list the two references [1] and [2] that outline this important 
result.?) 

3. The computational procedures required for linear optimal design may 
often be carried over to nonlinear optimal problems. For example, 

tReferences are located at the end of each chapter. 
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the nonlinear optimal design procedures based on the theory of the 
second variation [1-3] and quasilinearization [3, 4] consist of com- 
putational algorithms replacing the nonlinear problem by a sequence 
of linear problems. 

4. Linear optimal control designs turn out to possess a number of prop- 
erties, other than simply optimality, which classical control suggests 
are attractive. Examples of such properties are good gain margin 
and phase margin, and good tolerance of nonlinearities. This latter 
property suggests that controller design for nonlinear systems may 
sometimes be achieved by designing with the assumption that the sys- 
tem is linear (even though this may not be a good approximation), 
and by relying on the fact that an optimally designed linear system 
can tolerate nonlinearities — actually quite large ones — without 
impairment of all its desirable properties. Hence, linear optimal 
design methods are in some ways applicable to nonlinear systems. 

5. Linear optimal control provides a framework for the unified study 
of the control problems studied via classical methods. At the same 
time, it vastly extends the class of systems for which control designs 
may be achieved. 



1.2 ABOUT THIS BOOK IN PARTICULAR 

This is not a book on optimal control, but a book on linear optimal 
control. Accordingly, it reflects very little of the techniques or results of 
general optimal control. Rather, we study a basic problem of linear optimal 
control, the “regulator problem,” and attempt to relate mathematically all 
other problems discussed to this one problem. If the reader masters the 
mathematics of the regulator problem, he should find most of the remainder 
of the mathematics relatively easy going. (Those familiar with the standard 
regulator and its derivation may bypass Chapter 2, Sec. 2.1 through Chapter 
3, Sec. 3.3. Those who wish to avoid the mathematics leading to regulator 
results in a first reading may bypass Chapter 2, Sec. 2.2 through Chapter 3, 
Sec. 3.3.) 

The fact that we attempt to set up mathematical relations between the 
regulator problem and the other problems considered does not mean that 
we seek, or should seek, physical or engineering relations between the regula- 
tor problem and other problems. Indeed, these will not be there, and even 
the initial mathematical statements of some problems will often not suggest 
their association with the regulator problem. 

We aim to analyze the engineering properties of the solution to the 
problems presented. We thus note the various connections to classical con- 
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trol results and ideas, which, in view of their empirical origins, are often best 
for assessing a practical design, as distinct from arriving at this design. 



1.3 PART AND CHAPTER OUTLINE 



In this section, we briefly discuss the breakdown of the book into parts 
and chapters. There are five parts, listed below with brief comments. 

Part I — Introduction. This part is simply the introductory first chapter. 

Part II — Basic theory of the optimal regulator. These chapters serve to 
introduce the linear regulator problem and to set up the basic mathematical 
results associated with it. Chapter 2 sets up the problem, by translating into 
mathematical terms the physical requirements on a regulator. It introduces 
the Hamilton-Jacobi equation as a device for solving optimal control prob- 
lems, and then uses this equation to obtain a solution for problems where 
performance over a finite (as opposed to infinite) time interval is of interest. 
The infinite-time interval problem is considered in Chapter 3, which includes 
stability properties of the optimal regulators. Chapter 4 shows how to achieve 
a regulator design with a prescribed degree of stability. 

Part III — Properties and application of the optimal regulator. The aim 

of this part is twofold. First, it derives and discusses a number of engineering 
properties of the linear optimal regulator, and, second, it discusses the 
engineering implementation of the regulator. The main purpose of Chapter 5 
is to derive some basic frequency domain formulas and to use these to deduce 
from Nyquist plots properties of optimal systems involving gain margin, 
etc. In this chapter, the problem is also considered of designing optimal 
systems with prescribed closed-loop poles. In Chapter 6, an examination is 
made of the effect of introducing nonlinearities, including relays, into optimal 
systems. The main point examined is the effect of the nonlinearities on the 
system stability. Chapter 7 is mainly concerned with the effect of plant pa- 
rameter variations in optimal systems, and studies the effect using modern 
control ideas as well as the classical notion of the return difference. There is 
also further discussion in Chapter 7 of the design of optimal systems with 
prescribed closed-loop poles. Chapter 8 is devoted to the problem of state 
estimation; implementation of optimal control laws generally requires the 
feeding back of some function of the plant state vector, which may need 
to be estimated from the plant input and output if it is not directly measurable. 
The discussions of Chapter 8 include estimators that operate optimally in 
the presence of noise, the design of such estimators being achieved via solu- 
tion of an optimal regulator problem. The purpose of Chapter 9 is to tie 
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the estimation procedures of Chapter 8 with the optimal control results of 
earlier chapters so as to achieve controllers of some engineering utility. Atten- 
tion is paid to simplification of the structure of these controllers. 

Part IV — Extensions to more complex problems. In this part, the aim 
is to use the regulator results to solve a number of other linear optimal con- 
trol problems of engineering interest. Chapter 10 considers problems resulting 
in controllers using proportional-plus-integral state feedback. Chapter 11 
considers various versions of the classical servomechanism problem. Chapter 
12 considers problems when there is an upper bound on the magnitude of the 
control; the controllers here become dual mode, with one mode — the linear 
one — computable using the regulator theory. Next, Chapter 13 considers 
problems where only the plant output is available for use in a nondynamic 
controller, as well as other optimal problems that include controller con- 
straints. Such problems are often referred to as suboptimal problems. Chapter 
14 contains a very brief discussion of discrete time Systems, and continuous 
time-varying systems on an infinite-time interval. 

Part V — Computational aspects. This part — Chapter 15 — discusses 
some of the computational difficulties involved in carrying out an optimal 
control design. Various techniques are given for finding transient and steady 
state solutions to an equation, the matrix Riccati equation, occurring con- 
stantly in linear design. Approximate solutions are discussed, as well as a 
description of situations in which these approximate solutions are applicable. 

Appendices. Results in matrix theory and linear system theory relevant 
to the material in the book are summarized in the appendices. 
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CHAPTER 



THE STANDARD REGULATOR 

PROBLEM— I 



2.1 A REVIEW OF THE REGULATOR PROBLEM 

We shall be concerned almost exclusively with linear finite-dimensional 
systems, which frequently will also be time invariant. The systems may be 
represented by equations of the type 

x(t) = F(t)x(t) + G(t)u(t) (2.1-1) 

y(t) = H'(t)x(t). (2.1-2) 

Here, F(t ), G(t ), and H(t) are matrix functions of time, in general with con- 
tinuous entries. If their dimensions are respectively n X n, n x m, n x p, 
the n vector x(t) denotes the system state at time t 7 the m vector u(t) the system 
input or system control at time t, and the p vector y(t) the system output at 
time t . 

In classical control work, usually systems with only one input and out- 
put are considered. With these restrictions in (2.1-1) and (2.1-2), the vectors 
u(t) and y(t) become scalars, and the matrices G(t) and H{t) become vectors, 
and accordingly will often be denoted by lowercase letters to distinguish 
their specifically vector character. The systems considered are normally also 
time invariant , or stationary. In terms of (2.1-1) and (2.1-2), this means that 
the input u(t) and output y{t) for an initially zero state are related by a time- 
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invariant impulse response. Furthermore, the most common state-space 
descriptions of time-invariant systems are those where F(t ), g{t ), and h(t) 
are constant with time. (Note, though, that nonconstant F(t), g(t ), and h(t) 
may still define a time-invariant impulse response— e.g., F(t) = 0, g(/) = e\ 
h(t) = e _f defines the time-invariant impulse response e" Cf " T) .) 

The classical description of a system is normally in terms of its transfer 
function matrix, which we denote by W(s), s being the Laplace transform vari- 
able. The well-known connection between W{s) and the matrices of (2.1-1) 
and (2.1-2), if these are constant, is 

W(s) = H'(sl — F)~ l G. (2.1-3) 

A common class of control problems involves a plant , for which a control 
is desired to achieve one of the following aims: 

L Qualitative statement of the regulator problem. Suppose that initially 
the plant output, or any of its derivatives, is nonzero. Provide a 
plant input to bring the output and its derivatives to zero. In other 
words, the problem is to apply a control to take the plant from a 
nonzero state to the zero state. This problem may typically occur 
where the plant is subjected to unwanted disturbances that perturb 
its output (e.g., a radar antenna control system with the antenna 
subject to wind gusts). 

2. Qualitative statement of the tracking {or servomechanism) problem . 
Suppose that the plant output, or a derivative, is required to track 
some prescribed function. Provide a plant input that will cause this 
tracking (e.g., when a radar antenna is to track an aircraft such a 
control is required). 

In a subsequent chapter, we shall discuss the tracking problem. For the 
moment, we restrict attention to the more fundamental regulator problem. 

When considering the regulator problem using classical control theory, 
we frequently seek a solution that uses feedback of the output and its deriva- 
tives to generate a control. A controller , describable by a transfer function, 
is interposed between the plant output and plant input, with the plant output 
constituting the controller input. The controller output is the plant input. 
In Fig. 2.1-1, the feedback arrangement is shown. Both the plant and con- 
troller have a single input and output, and are time invariant — i.e., possess 
a transfer function. 

In the modern approach, it is often assumed that the plant states are 
available for measurement. (If this is not the case, it is generally possible to 
construct a physical device called a state estimator , which produces at its 
output the plant states, when driven by both the plant input and output. 
This will be discussed in a later chapter.) In addition to assuming availability 
of the states, the design of controllers is often restricted by a requirement 
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Fig. 2.1-1 Classical feedback arrangement. 



that they be nondynamic, or memoryless. In other words, the controller 
output or plant input u(t) is required to be an instantaneous function of the 
plant state x(t). The nature of this function may be permitted to vary with 
time, in which case we could write down a control law 

u(t) = k(x(t), t) (2.1-4) 

to indicate the dependence of u(t) on both x(r) and /. 

Of interest from the viewpoint of ease of implementation is the case of 
the linear control law , given by 

u(t) = K'(t)x(t) (2.1-5) 

for some matrix K of appropriate dimension. [When K(t) is a constant 
matrix, (2.1-5) becomes a constant or time-invariant control law, and, as 
will become clear, a number of connections with the classical approach can 
be made.] Figure 2.1-2 shows the modern control arrangement. The plant is 
assumed to be linear, but may have multiple inputs and outputs and may be 
time varying. The state estimator constructs the plant state vector from the 




Fig. 2.1-2 Modern control feedback arrangement. 
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input and output vectors, and is actually a linear, finite-dimensional system 
itself. Linear combinations of the states are fed back to the system input in 
accordance with (2.1-5). 

When attempting to construct a controller for the regulator problem, 
we might imagine that the way to proceed would be to search for a control 
scheme that would take an arbitrary nonzero initial state to the zero state, 
preferably as fast as possible. Could this be done? If Fand G are constant, 
and if the pair [F, G] is completely controllable , the answer is certainly yes 
[1]. Recall (see Appendix B), that the definition of complete controllability 
requires that there be a control taking any nonzero state x(t 0 ) at time t 0 
to the zero state at some time F. In fact, il Fand G are constant, T can be 
taken as close to / 0 as desired, and for some classes of time-varying F(t) and 
G(t), this is also the case. What, therefore, would be wrong with such a 
scheme? Two things. First, the closer T is to t 0 the greater is the amount of 
control energy (and the greater is the magnitude of the control) required to 
effect the state transfer. In any engineering system, an upper bound is set on 
the magnitude of the various variables in the system by practical considera- 
tions. Therefore, one could not take T arbitrarily close to / 0 without exceed- 
ing these bounds. Second, as reference to [l] will show, the actual control 
cannot be implemented as a linear feedback law, for finite T, unless one is 
prepared to tolerate infinite entries in K(T) — i.e., the controller gain at time 
T. In effect, a linear feedback control is ruled out. 

Any other control scheme for which one or both of these objections 
is valid is equally unacceptable. In an effort to meet the first objection, one 
could conceive that it is necessary to keep some measure of control magni- 
tude bounded or even small during the course of a control action; such 
measures might be 

f u'(t)u(t)dt 9 f [u'(t)u(t)] [il dt, max || «(/)(), 

J ‘0 J fo /f l/o, T] 

or f u'(t)R(t)u(t ) dt 

J /o 

where the superscript prime denotes matrix (or vector) transposition, and 
R(t) is a positive definite matrix for all t, which, without loss of generality, 
can always be taken as symmetric. We shall discuss subsequently how to meet 
the second objection. Meanwhile, we shall make further adjustments to our 
original aim of regulation. 

First, in recognition of the fact that “near enough is often good enough” 
for engineering purposes, we shall relax the aim that the system should actu- 
ally achieve the zero state, and require merely that the state as measured by 
some norm should become small. If there were some fixed time T by which 
this was required, we might ask that x'(T)Ax(T), with A some positive definite 
matrix, be made small. 
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Second, it is clearly helpful from the control point of view to have 
\\x(t)\\ small for any t in the interval over which control is being exercized, 
and we can express this fact by asking (for example) that x'(t)Q(t)x(t) dt 
be small, where Q(t ) is symmetric positive definite. In some situations, as 
we shall see, it proves sufficient to have Q(t) nonnegative definite. 

Let us now sum up the desirable properties of a regulator system. 

1. The regulator system should involve a linear control law, of the form 
u(t) = K'(t)x(t). 

2. The regulator scheme should ensure the smallness of quantities such 
as \ T to u\t)R(t)u(t) dt , x'(T)Ax(T), and J7 o x\t)Q{t)x{t) dt , where 7?(-), 
A, and Q(-) have the positivity properties mentioned earlier. 

For the moment, we shall concentrate on 2. Subsequently, we shall see 
that a regulator scheme with property 2 automatically possesses property 1 
too. 

To consider achieving property 2, let us define the performance index 
V(x(tf), u(-), t 0 ) — f (u f Ru + x'Qx) dt + x'{T)Ax{T). (2.1-6) 

J to 

As the notation implies, the value taken by V depends on the initial state 
x(t Q ) and time t 0 , and the control over the interval [f 0 , T]. 

A reasonable approach to achieving property 2 would be to choose that 
control u(-) which minimized the performance index (2.1-6) [always assuming 
such a u(-) exists]. This idea leads to the following formal statement of the 
regulator problem. 

Regulator problem . Consider the system (2.1-1), where the entries of 
F(t) and G(t) are assumed to be continuous. Let the matrices Q{t) and 
R(t) have continuous entries, be symmetric, and be nonnegative and 
positive definite, respectively. Let A be a nonnegative definite symmetric 
matrix. Define the performance index K(x(/ 0 ), w(«), / 0 ) as in (2.1-6) and 
the minimization problem as the task of finding an optimal control 
w *(0> minimizing K, and the associated optimum performance 

index F*(x(/ 0 ), / 0 ) — i.e., the value of V obtained by using the optimal 
control. 

Notice that earlier it was suggested that A should be positive definite, 
whereas the statement of the regulator problem merely suggests that it should 
be nonnegative definite. As we shall see subsequently, the size of the final state 
x(T) can frequently be made small merely by the relaxed requirement. 
Indeed, the choice A = 0 will often lead to a satisfactory result. 

As already remarked, the minimization of (2.1-6) turns out to be achiev- 
able with a linear feedback law. This is the reason why the performance 
index includes a measure of control energy in the form u'(t)R(t)u(t) dt 9 
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rather than, say, \l [w'OMO] 5 dt, or any of the other measures suggested. 
These other measures do not, in general, lead to linear feedback laws. The 
form of the other terms in the performance index (2.1-6) is also generally 
necessary to obtain linearity of the feedback law. 

Before studying the minimization problem further, we note the following 
references. Books [2] and [3] are but two of a number of excellent treatments 
of the regulator problem, and, of course, optimal control in general. Several 
papers dealing with the regulator problem could be read with benefit [4-7]. 

Problem 2.1 -1 . Consider the system 

x = F(t)x + G{t)u 

with F, G possessing continuous entries. Show that there does not exist a control 
law 

u — K'(t)x(t) 

with the entries of K(t ) continuous, such that with arbitrary x(t 0 ) and some finite 
T, x(T) — 0. [Hint: Use the fact that if x — F{t)x where F has continuous entries 1 
then a transition matrix exists.] ^ 

Problem 2.1 -2. Electrical networks composed of a finite number of intercon- 
nected resistors, capacitors, inductors, and transformers can normally be describe^ 
by state-space equations of the form t : 

x = Fx + Gu 

y — H'x + Ju. 

The entries of the state vector will often correspond to capacitor voltages and 
inductor currents, the entries of the input vector to the currents at the various ports 
of the network, and the entries of the output vector to the voltages at the various 
ports of the network. Assuming the initial x{t 0 ) is nonzero, give a physical inter- 
pretation to the problem of minimizing 

f ( u'Ju -f- x'Hu) dt. 

J to 



2.2 THE HAMILTON-JACOBI EQUATION 

In this section, we temporarily move away from the specific regulator 
problem posed in the last section to consider a wider class of optimization 
problems requiring the minimization of a performance index. We shall, in 
fact, derive a partial differential equation, the Hamilton- Jacobi equation , 
satisfied by the optimal performance index under certain differentiability 
and continuity assumptions. Moreover, it can be shown that if a solution 
to the Hamilton-Jacobi equation has certain differentiability properties, 
then this solution is the desired performance index. But since such a solution 
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need not exist, and not every optimal performance index satisfies the Hamil- 
ton-Jacobi equation, the equation only represents a sufficient, rather than a 
necessary, condition on the optimal performance index. 

In this section, we shall also show how the optimal performance index, 
if it satisfies the Hamilton-Jacobi equation, determines an optimal control. 
This will allow us in Sec. 2.3 to combine the statements of the regulator 
problem and the Hamilton-Jacobi theory, to deduce the optimal performance 
index and associated optimal control for the regulator problem. 

Other approaches will lead to the derivation of the optimal control and 
optimal performance index associated with the regulator problem, notably 
the use of the Maximum Principle of Pontryagin, combined with the Euler- 
Lagrange equations, [2], [3], and [8]. The Maximum Principle and Euler- 
Lagrange equations are lengthy to derive, although their application to the 
regulator problem is straightforward. The simplest route to take without 
quoting results from elsewhere appears to be the development of the Hamil- 
on-Jacobi equation with subsequent application to the regulator problem. 
\ctually, the Hamilton-Jacobi equation has so far rarely proved useful except 
or linear regulator problems, to which it seems particularly well suited. 

The treatment we follow in deducing the Hamilton-Jacobi equation is 
blend of treatments to be found in [3] and [7]. We start by posing the follow- 
ng optimal control problem. For the system 

x = f(x, u, t) x(t Q ) given (2.2-1) 

ind the optimal control w*(r), te[t 0 , T], which minimizes 

w(.), t 0 ) = f /( jc(t), u( t), r) dr + m(x(T)). (2.2-2) 

J to 

Without explicitly defining for the moment the degree of smoothness — i.e., 
the number of times quantities should be differentiable — we shall restrict/, 
/, and m to being smooth functions of their arguments. Otherwise, f(x, u , t) 
can be essentially arbitrary, whereas I(x(r), u( r), r) and m(x(T)) will often 
be nonnegative, to reflect some physical quantity the minimization of which 
is desired. As the notation implies, the performance index depends on the 
initial state x(t 0 ) and time t 0 , and the control u{t) for all te[t 0l T]. The optimal 
control «*(•) may be required a priori to lie in some special set, such as the 
set of piecewise continuous functions, square-integrable functions bounded 
by unity, etc. 

Let us adopt the notation u [a>b] to denote a function w(«) restricted to the 
interval [a, b]. Let us also make the definition 

v *(x(t), t) - min V(x(t) y w(.), 0* (2.2-3) 

u[t,T ] 

That is, if the system starts in state x(t) at time t , the minimum value of the 
performance index (2.2-2) is F*(x(/), t). Notice that V*(x (t), t) is independent 
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of u( ■). precisely because knowledge of the initial state and time abstractly 
determines the particular control, by the requirement that the control 
minimize V(x(t), u( ■ ), t). Rather than just searching for the control minimiz- 
ing (2.2-2) and for the value of K*(x(f 0 ), ?„) for various x(t 0 ), we shall study 
the evaluation of (2.2-3) for all t and x(t), and the determination of the asso- 
ciated optimum control. Of course, assuming we have a functional expression 
for V* in terms of x(t ) and t, together with the optimal control, we solve the 
optimization problem defined by (2.2-1) and (2.2-2) by setting t = t 0 . 

Now, for arbitrary t in the range [? 0 , T] and f, in the range [t, T], we have 

V*(x(t), t ) = min f l(x(x), u( t), t) dx + m{x{T)) 

ult.ri Ly i 

= min (min f !(x( t), u(t), t) dx 

ult.ti) (ulti.n L^t 



+ f l(x( t), u(t\ x) dx + m(x(T)) > 

J ti J, 

= min { f /(jt(t), u(x ), t) dx 

ult,t ,) l J t 

T ^ 

+ min f l(x( x), u(t), x) dx + m(x(T)) 1 
uin.ri L J fi JJ 



or 



V*(x(t), t) = min f f l(x(x). u( r), t) dx + V*(x(t l ), /,) . (2.2-4) 

u[t,t i) Ly t J 

In (2.2-4), let us now set /,=/-(- At, where At is small. Applying Tay- 
lor’s theorem to expand the right-hand side (noting that the smoothness 
assumptions permit us to do this), we obtain 

V*(x(t), t)= min j At l(x(t + a At), u(t + a At), t + a At) 

uU,t-hAl) . 

+ V*(x(t), t) + [^ (*(/), oj ~j~At 
+ W- (*(')> OA/ + 0(A/) 2 j 

where a is some constant lying between 0 and 1. Immediately, 
dV* ( 

- 5 -(x(f), t ) = -min j/(x(t + a At), u(t + a At), t + tx At) 

u[t,t + At) { 7 J 



+ 



r dV* 



^ ^ i x (0> 0 f(x(t ), u(t) f t) -}- 0 (At) 



Now, let At approach zero, to conclude that 



dV* 

0 = —mm { 



U(t) 



l(x(t), u(t), t) + ^ (x(t), t) 



)] /(*('), M (t), t) 



> • 
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In this equation, /and / are known functions of their arguments, whereas V* 
is unknown. In order to emphasize this point, we shall rewrite the equation as 



dV* 

dt 



— min 

u(0 



l(x(t), u(t\ 0 + 



dV*' n 

u{t), t) . 



(2.2-5) 



This is one statement of the Hamilton-Jacobi equation. In this format, it 
is not precisely a partial differential equation but a mixture of a functional 
and a partial differential equation. 

The value of u(t) minimizing the right-hand side of (2.2-5) will depend 
on the values taken by x(t ), dV*/dx, and t. In other words, the minimizing 
u(t) is an instantaneous function — call it u(x(t\ dV*/dx , t) — of the three vari- 
ables x(t), dV*jdx, t. [To be sure, if the explicit way dV*jdx depended on 
x(t) and t were known, at this point one could express the minimizing u(t) 
as a function of merely x(t) and /. But at this point, the explicit functions that 
K*, and dV*/dx, are of x(r) and t have yet to be found.] 

The preceding derivations also tell us that to minimize V(x{t), u ( •), t), 
the value of the minimizing control at time t is precisely u(x(t) y dV*jdx, t). 

With the above definition of w(», -, •), Eq. (2.2-5) becomes 

u(x(t), I), t ] 



dV*' 

dx 



f[(x(t), u(x(i), 




( 2 . 2 - 6 ) 



Despite the bewildering array of symbols within symbols, (2.2-6) is but a 
first-order partial differential equation with one dependent variable, V*, 
and two independent variables, x(t) and t, because /,/, and u are known func- 
tions of their arguments. 

A boundary condition for (2.2-6) is very simply derived. Reference to 
the performance index (2.2-2) shows that V(x(T), w(-), T) — m(x(T)) for all 
u(-), and, accordingly, the minimum value of this performance index with 
respect to u(-) is also m(x(T)). That is, 

V*(x(T), T ) = m(x{T)). (2.2-7) 

The pair (2.2-6) and (2.2-7) may also be referred to as the Hamilton-Jacobi 
equation, and constitute a true partial differential equation. 

If the minimization implied by (2.2-5) is impossible— i.e., if £(•, •, •) 
does not exist— then the whole procedure is invalidated and the Hamilton- 
Jacobi approach cannot be used in tackling the optimization problem. 

We now consider how to determine the optimal control for the problem 
defined by (2.2-1) and (2.2-2). We assume that (2.2-6) and (2.2-7) have been 
solved so that V*(x(t), t) is a known function of x(t) and t. Now, V*(x(t), t) 
determines a function of x(t) and z — call it u(x(t), r)— via the formula 

1 /* 

«W0, t) = w[x(r), ^j(x(0, 0, t]. 



( 2 . 2 - 8 ) 
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That is, u is the same as u , except that the second variable on which u depends 
itself becomes a specified function of the first and third variables. 

This new function u(*, • ) has two important properties. The first and 
more easily seen is that w(x(f), 0 is the value at time t of the optimal control 
minimizing 

V(x{t), «(•)> 0 = J l(x( t), u( t), t) dr + m(x(T )). (2.2-9) 

That is, to achieve the optimal performance index V*(x(t ), t ), we start off 
with a control u(x(t), /). That this is so is implicit in the arguments leading 
to (2.2-5), as previously shown. 

The second property is that the optimal control w*(-) for the original 
minimization problem defined by (2.2-1) and (2.2-2)— with t 0 as the initial 
time and t as an intermediate value of time — is related to w(*, •) simply by 

u*(t ) = u(x(t),t) (2.2-10) 

when x(?) is the state at time t arising from application of w*(*) over f t 0 , t). 

To some, this result will be intuitively clear. To demonstrate it, we exam- 
ine a variant of the arguments leading to (2.2-4). By definition, 



V*(x(t 0 ), ? 0 ) = min [ f l(x( t), u( t), t) dx + m(x(Tj) » 

and the minimum is achieved by w*( •). With w*(-) regarded as the sequential 
use of u* [tQit) and and with the assumption that u ltQ t) is applied until time 
t , evidently 



K*(x(? 0 ), t 0 ) — min f /(x(t), w(t), t) dx + w(x(T)) (2.2-11) 

u[t, T ) fo _ 



= f l(x( r), u(x), x) dx 

J to 

+ min | f l(x{x), u(x ), r) + m(x(T)) 

u[l,T] LJ t 



( 2 . 2 - 12 ) 



The minimization in (2.2-11), and therefore (2.2-12), is achieved by 
In other words, u% n is the optimal control for the system (2.2-1) with per- 
formance index 



t(x(r), «(t), t) dx + m(x(T)) (2.2-13) 

d t 

with initial state x(t), where x(t) is derived by starting (2.2-1) at time t 0 in 
state x(t 0 ), and applying (This is, in fact, one statement of the Principle 

of Optimality, [10], to the effect that a control policy optimal over an interval 
[t 0 , T] is optimal over all subintervals [t, T].) But u(x(r), t) is the value of the 
optimal control at time t for the performance index (2.2-13), and so 

u(x(t), t) = ul„(t) = «*(/). (2.2-14) 
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Several points should now be noted. First, because of the way it is cal- 
culated u(x{t), t ) is independent of / 0 . The implication is, then, that the 
optimal control at an arbitrary time t for the minimization of 

V(x(cr ), w(*), c) = [ l(x( t), u( t), t) dr + m(x(T)) (2.2-15) 

is also u(x(t ), I). Put another way, the control w(x(*), •) is the optimal control 
for the whole class of problems (2.2-15), with variable x(<r) and a. 

The second point to note is that the optimal control at time t is given 
in terms of the state x(t) at time /, although, because its functional depen- 
dence on the state may not be constant, it is in general a time-variable 
function of the state. It will be theoretically implementable with a feedback 
law, as in Fig. 2.2-1. (Other schemes, such as the Maximum Principle and 




Fig. 2.2-1 Feedback implementation of the optimal control. 

Euler-Lagrange equations, for computing the optimal control do not neces- 
sarily have this useful property; in these schemes, the optimal control may 
often be found merely as a certain function of time.) 

The third point is that the remarks leading to the Hamilton-Jacobi 
equation are reversible, in the sense that if a suitably smooth solution of the 
equation is known, this solution has to be the optimal performance index 
V*(x(t), /). 

Finally, rigorous arguments, as in, for example, [4] and [5], pin down 
the various smoothness assumptions precisely and lead to the following 

conclusion, which we shall adopt as a statement of the Hamilton-Jacobi 

results. 

* 



Hamilton-Jacobi equation. Consider the system 

X = fix, u, t) 
and the performance index 



V(x(t), «(•), t ) 



* r 
J t 



Kx(t), u(t), t) dr + m(x(T)). 



(2.2-1) 

(2.2-9) 
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Suppose /, /, and m are continuously differentiable in all arguments, 
that there exists an absolute minimumt of l(x , w, t ) + X' f(x, w, t) with 
respect to u(t) of the form u(x(t ), X, t), and that u is continuously differ- 
entiable in all its arguments. Furthermore, suppose that V*(x(t ), /) is a 
solution of the Hamilton- Jacobi equation (2.2-5) or (2.2-6) with 
boundary condition (2.2-7). Then F*(*, •) is the optimal performance 
index for (2.2-9), and the control given by (2.2-8) is the optimal control 
at time t for the class of problems with performance index 

F(x(<j), m(.), a) - f /(x(z), u{ t), t) dx + m(x{T)\ (2.2-15) 

Conversely, suppose that /, /, and m are continuously differentiable 
in all arguments, that there exists an optimal control, and that the corre- 
sponding minimum value of (2.2-9), V*(x(t), /), is twice continuously 
differentiable in its arguments. Suppose also that l(x, u, t) + ( dV*/dx )' 
f(x, u, t ) has an absolute minimum with respect to u(t) at u(x(t), t), and 
that «(•, •) is differentiable in x and continuous in t. Then V*(-, •) 
satisfies the Hamilton-Jacobi equation (2.2-5) or (2.2-6) with boundary 
condition (2.2-7). 



We conclude this section with a simple example illustrating the deriva- 
tion of the Hamilton-Jacobi equation in a particular instance. Suppose we 
are given the system 

x = u, 

with performance index 



F(x(0), «(•), 0) = f (u 2 + x 2 + |x 4 ) dt. 

J 0 

Using Eq. (2.2-5), we have 



dV* . / 

—3— = —mm < 
dt «(/) l 

The minimizing u{t) is clearly 



u 2 + x ; 






+ - 



dV‘ 



dx 




and we have 



u = 



1 dV* 

2 dx ’ 



dV* 

dt 



W dV*V 
4 Wx ) 




as the Hamilton-Jacobi equation for this problem, with boundary condition 
V (x(T\ T ) = 0. The question of how this equation might be solved is 



tThough the point is inessential to our development, we should note that u{t) may 
be constrained a priori to lie in some set U(t) strictly contained in the Euclidean space of 

dimension equal to the dimension of u. All minimizations are then subject to the con- 
straint u(t ) e £/(/). 
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quite unresolved by the theory presented so far. In actual fact, it is rarely 
possible to solve a Hamilton-Jacobi equation, although for the preceding 
example, a solution happens to be available [1 1]. It is extraordinarily com- 
plex, and its repetition here would serve no purpose. 

Problem 2.2-1 . Consider a system of the form 

x = f{x) -f gu 

with performance index 

V{x{t\ w(-), t) = £ (« 2 + g(x)) dt . 

Show that the Hamilton-Jacobi equation is linear in dV*jdt and quadratic in 
dV*/dx. 

Problem 2.2-2. Let u be an r vector, and let X and x be n vectors. Let A, B y C 
be constant matrices of appropriate dimensions such that the following function of 
u , x, and p can be formed: 

Q(u, x , p) = u'Au -f- Ix'Bu -f 2u Cp. 

Show Q has a unique minimum in u for all x and p if, and only if, \(A 4- A') is 
positive definite. 



2.3 SOLUTION OF THE FINITE-TIME REGULATOR 
PROBLEM 

In this section, we return to the solution of the regulator problem, 
which we restate for convenience. 

Regulator problem . Consider the system 

x = F(t)x(t) + G{t)u(t) x(t 0 ) given (2.3-1) 

with the entries of F(f), G(r) assumed continuous. Let the matrices 
Q(t) and R(t) have continuous entries, be symmetric, and be non- 
negative and positive definite, respectively. Let A be a nonnegative 
definite matrix. Define the performance index , 

V(x(tal «(•), to) - r (u'Ru + x'Qx ) dt + x'(T)Ax(T\ (2.3-2) 

and the minimization problem as the task of finding an optimal control 

u (0> te[to, r], minimizing V and the associated optimum performance 
index K*(x(/ 0 ), /„). 

For the moment, assume that Jis finite. 

To solve the problem, we shall make use of the results on the Hamilton- 
Jacobi equation summarized at the end of the last section. An outline of the 
problem solution follows. 
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1. We shall show by simple arguments independent of the Hamilton- 
Jacobi theory that the optimal performance index V*(x(t ), t), if it 
exists, must be of the form x'(t)P(t)x(t ), where P(t) is a symmetric 
matrix. 

2. With the assumption that F*(x(t), t) exists, result 1 will be used 
together with the Hamilton-Jacobi theory to show that P{t) satisfies 
a nonlinear differential equation — in fact, a matrix Riccati equation. 

3. We shall establish existence of V*{x{t ), /). 

4. We shall find the optimal control. 

To carry out this program, it is necessary to make the following tempo- 
rary assumption. 

Temporary Assumption 2.3-1. Assume that F(t ), G(f), /?(*), and Q{t) have 
entries that are continuously differentiable. 

This assumption is removed in Prob. 2.3-2. 

We note that some treatments of the regulator problem assume a priori 
the form x'(t)P(t)x(t) for the optimal performance index. It is therefore 
interesting to observe a simple derivation of this form appearing in [12]. 

The quadratic form of V*(x(t), t). The necessary and sufficient conditions 
for V*(x{t), t) to be a quadratic form are that 

V*(Ax, t) = A 2 V*(x , t) for all real A (2.3-3) 

t) + V*(x 2 , t) = U v *(xt + x 29 1 ) + V*(x 1 - x 29 ()]. (2.3-4) 

[The student is asked to verify this claim in Prob. (2.3-1).] To show that 
(2.3-3) and (2.3-4) hold, we adopt the temporary notation u* to denote the 
optimal control over [t, T] when the initial state is x{t) at time t . Then the 
linearity of (2.3-1) and the quadratic nature of (2.3-2) imply the following 
equalities, whereas the inequalities follow directly from the fact that an 
optimal index is the minimum index. We have that 

V*(Ax, t) < V(Ax, h /*(.), t) = A 2 V*(x , t) 

PV*(x, t) < A 2 V(x, t) = V*(Ax, t ) 

for all real constants A. These imply (2.3-3) directly. Similar reasoning gives 
the inequality 

V*(x u t) + V*(x 2> t) = \{V% 2x„ t) + V*(2x 2 , t )] 

< 1[F(2 x,,u* +Xl + u*. x „ t) 

+ V(2x 2 , u* +Xl - u*. Xl , ?)] 

= i[F(x, + x 2 , u* t ) 

+ V(x { - x 2 , u*. x „ 0] 

= + x 2 , t) + V*(x t - x 2 , /)]. (2.3-5) 
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By making use of the controls w* and w* , we establish the following inequality 
in a like manner: 

\[V*(x, + 0 + V*( Xl - x 2 , t)] < V*(x l9 1 ) + V*(x 2 , t). (2.3-6) 

Then (2.3-5) and (2.3-6) imply (2.3-4). 

We conclude that V*(x(t), t) has the form 

V*(x(t\ t) = x'(t)P(t)x(t) (2.3-7) 

for some matrix P(t) 9 without loss of generality symmetric. {If P(t) is not 
symmetric, it may be replaced by the symmetric matrix %[P(t) + P'(01 without 
altering (2.3-7)}. 



Derivation of the matrix Riccati equation. Now we shall show, using the 
Hamilton-Jacobi equation, that the symmetric matrix P(t) satisfies a matrix 
Riccati equation. 

The first form of the Hamilton-Jacobi equation is now repeated: 



dV*(x(t\ t) 
dt 



= —min { 

u(t ) 



u(t), t) + 



r 



dV* 



-gjr (*(')> OJ /(*( 0, M», 0} 



(2.3-8) 

In our case, /(x(r), */(/), /) is u' (t)R(t)u(t) + x'(t)Q(t)x(t)i [(dV*ldx)(x(t), t)]' 
from (2.3-7) is 2x'(t)P(t), whereas f(x(t), u(t ), r) is simply F(t)x(t) + G(t)u(t). 
The left side of (2.3-8) is simply x'(t)P(t)x(t). Hence, Eq. (2.3-8) becomes, in 
the special case of the regulator problem, 

x'Px ~ —min [u'Ru + x'Qx + Ix'PFx + 2 x'PGu]. (2.3-9) 

u(f) 

To find the minimum of the expression on the right-hand side of 
(2.3-9), we simply note the following identity, obtained by completing the 
square: 

u'Ru + x'Qx + Ix'PFx + Ix'PGu = (u + R~ { G'Px) f R(u + R~ l G'Px ) 



+ x'(Q - PGR-'G'P 



+ PF + F'P)x . 

Because the matrix R(t) is positive definite, it follows that (2.3-9) is minimized 
by setting 

u(t) - -R~\t)G'(t)P(t)x(t) y (2.3-10) 

in which case one obtains 



x'(t)P(t)x{t) = -x'(t)[Q(t) - P(t)G(t)R-'(t)G’(t)P(t) 

+ P(t)F(t) + F'(t)P(t)]x(t). 

Now this equation holds for all x(t); therefore, 

~P(t) = P(t)F(t) + F'(t)P(t) - P(t)G(t)R-'(t)G'(t)P(t) + Q(t) (2.3-1 1) 
where we use the fact that both sides are symmetric. 
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Equation (2.3-11) is the matrix Riccati equation we are seeking. It has 
a boundary condition following immediately from the Hamilton-Jacobi 
boundary condition. We recall that V*(x(T ), T ) = m(x(T)), which implies 
in the regulator problem that x'(T)P(T)x(T) = x'(T)Ax(T). Since both P(T) 
and A are symmetric, and x(T) is arbitrary, 

P(T) = A. (2.3-12) 

Before we proceed further, it is proper to examine the validity of the 
preceding manipulations in the light of the statement of the Hamilton- 
Jacobi equation at the end of the last section. Observe the following. 

1. The minimization required in (2.3-9) is, in fact, possible, yielding the 
continuously differentiable minimum of (2.3-10). [In the notation of 
the last section, the expression on the right of (2.3-10) is the function 
w(*, • , •) of x(t), dV*jdx, and t .] 

2. The loss function x'Qx + u'Ru and function Fx + Gu appearing in 
the basic system equation have the necessary differentiability prop- 
erties, this being guaranteed by Temporary Assumption 2.3-1. 

3. If P(t), the solution of (2.3-1 1), exists, both P(t) and P(t) exist and are 
continuous, the former because of the relation (2.3-11), the latter 
because differentiation of both sides of (2.3-11) leads to P{t) being 
equal to a matrix with continuous entries (again, Temporary Assump- 
tion 2.3-1 is required). Consequently, x\t)P(t)x{t) is twice continu- 
ously differentiable. 

Noting that Eqs. (2.3-11) and (2.3-12) imply the Hamilton-Jacobi 
equation (2.3-8), with appropriate initial conditions, we can then use the 
statement of the Hamilton-Jacobi equation of the last section to conclude 
the following. 

1. If the optimal performance index F*(x(/), /) exists, it is of the form 
x'(t)P(t)x(t), and P(t) satisfies (2.3-11) and (2.3-12). 

2. If there exists a symmetric matrix P(t) satisfying (2.3-1 1) and (2.3-12), 
then the optimal performance index V*{x(t), t) exists, satisfies the 
Hamilton-Jacobi equation, and is given by x\t)P{t)x(t ). 

In theory, P(t\ and in particular P(t Q ), can be computed! from (2.3-11) 
and (2.3-12). Thus, aside from the existence question, the problem of finding 
the optimal performance index is solved. 

Existence of the optimal performance index V*(x(t),t). Here we shall 
argue that V*(x(t), t) must exist for all t < T. Suppose it does not. Then, 

fBy the word computed, we mean obtainable via numerical computation, generally 
by using a digital computer. There is no implication that an analytic formula yields 
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by the preceding arguments, Eqs. (2.3-1 1) and (2.3-12) do not have a solution 
P(t) for all t < r. 

The standard theory of differential equations yields the existence of a 
solution of (2.3-11) and (2.3-12) in a neighborhood of T. For points suffi- 
ciently far distant from T 7 , a solution may not exist, in which case (2.3-11) 
exhibits the phenomenon of a finite escape time. That is, moving back earlier 
in time from T, there is a first time T' such that P(t) exists for all t in ( T, T], 
but as t approaches T\ some entry or entries of P(t) become unbounded. 
Then P(t) fails to exist for t < T\ Moreover, the only way that the solution 
of (2.3-1 1) and (2.3-1 2) can fail to exist away from T is if there is a finite escape 
time. 

Since our assumption that F*(x(0> 0 does not exist for all t < T implies 
that there is a finite escape time, we shall assume existence of a finite escape 
time T' < T and show that this leads to a contradiction. We have V*(x(t) 9 t) 
exists for all / in (T\ T], and, in particular, V*(x(T* + e) 9 T + €) exists for 
all positive e less than (T — T ). Now 

0 < V*(x(T f + e) 9 r + e) = x\T' + e)P{T + e)x(T' + e), 

the inequality holding because of the nonnegativity of both the integrand 
for all u and the final value term in (2.3-2). Hence, P(V + e) is a nonnegative 
definite matrix. As 6 approaches zero, some entry becomes unbounded; 
without loss of generality, we can conclude that at least one diagonal entry 
becomes unbounded. If this were not the case, a certain 2x2 principal 
minor of P(T f + e) must become negative as e approaches zero, which con- 
tradicts the nonnegative definite property of /XT 7 ' + e). Therefore, we suppose 
that a diagonal element — say, the /th — is unbounded as e approaches zero; 
let e t be a vector with zeros for all entries except the /th, where the entry is 1. 
Then 

T' + €) = Pii (T' + 6), 

which approaches infinity as e approaches zero. (Here, p i} denotes the entry 
in the /th row and /th column of P .) 

But the optimal performance index is never greater than the index result- 
ing from using an arbitrary control. In particular, suppose the zero control 
is applied to the system (2.3-1), and let 0(/, t) denote the transition matrix. 

Starting in state <? t . at time T + 6, the state at time t is 0(x, T + e)e if 
and the associated performance index is 

v(e t9 0, r +€)= [ T ep'ir, T + e)Q( t)<D(t, T + e)e t . dx 

^ r+t 

+ e$\T, T + €)A<b(T, T + e)e i9 

which must not be smaller than Pn{T' +6). But as 6 approaches zero, 
V(e i9 0, T' + f) plainly remains bounded, whereas p w (T' + 0 approaches 
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infinity. Hence, we have a contradiction that rules out the existence of a finite 
escape time for (2.3-11). 

Thus, (2.3-1 1) and (2.3-12) define jP(f) for all / < T, and therefore the index 
F*(x(t), t) = x’(t)P(t)x(t) exists for all t < T. 

The optimal control. In the course of deriving the Riccati equation, 
we found the optimal control at time t for the regulator problem with initial 
time t when constructing the minimizing u(t) of (2.3-9) in Eq. (2.3-10). But, 
as pointed out in the last section, this gives the optimal control u*(-) for an 
arbitrary initial time [see (2.2-10)]; thus, 

«*(/) - - R ~ 1 (t)G’(t)P(t)x(t). (2.3- 1 3) 

Note that in (2.3-10), P(t) was unknown. The product 2 P(t)x(t) represents 
dV*/dx, and u(t) was to be regarded as being defined by independent variables 
x(t) (actually absent from the functional form for w), dV*/dx, and t. Subse- 
quently, we were able to express dV*jdx explicitly in terms of t and x, since 
P(t) became explicitly derivable from (2.3-1 1). This led to the feedback law 
of (2.3-13). Notice, too, that Eq. (2.3-1 3) is a linear feedback law, as promised. 

Problem 2.3-2 allows removal of Temporary Assumption 2.3-1, and, 
accordingly, we may summarize the results as follows. 

Solution of the regulator problem. The optimal performance index 
for the regulator problem with initial time t and initial state x(t) is 
x’(t)P(t)x(t), where P(t) is given by the solution of the Riccati equation 
(2,3-11) with initial condition (2.3-12). The matrix P(t) exists for all 
/ < T. The optimal control for the regulator problem with arbitrary 
initial time is given by the linear feedback law (2.3-13) for any time t 
in the interval over which optimization is being carried out. 

To illustrate the previous concepts, we consider the following example. 
The system equation is 

x = \x + u 

and the performance index is 

f (2 e~'u 2 -f ?e~‘x 2 ) dt. 

J to 

We require, of course, the optimal control and associated optimal perfor- 
mance index. The Riccati equation associated with this problem is 

~P =P - ±e‘P 2 + P(T) - 0. 

The solution of this equation may be verified to bet 

P(t) = (1 — e c e~ T )(e‘ + e 2t e ~ T )~ l . 



f Later, we shall discuss techniques for deriving analytic solutions for Riccati equations. 
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The optimal control is thus 

u(t) = — ^(1 — e'e' r )(l + e ( e~ T )~ l x(t) 
and the optimal performance index is 

F*(x(/ 0 ), t 0 ) = x'Oo)(l - e to e~ T )(e to + e lto e^ T )' l x(t 0 ). 

Problem 2.3-1 . Show that Eqs. (2.3-3) and (2.3-4) imply the existence of a 
matrix P(t) such that V*(x(t) y t ) = x'(t)P(t)x(t). [Hint: Without loss of generality. 
Pit) is symmetric. Use * = e t in (2.3-3) to define the diagonal entries of P(t) 9 and 
then specific values of x { and x 2 in (2.3-4) to define the off-diagonal entries of P(t). 
With P(t) so defined, check that the definition is compatible with (2.3-3) and (2.3-4) 
holding for all x, x u and x 2 -] 

Problem 2.3-2. Consider the regulator problem posed at the start of the section, 
without Temporary Assumption 2.3-1 , and let P(t) be defined by (2.3-1 1 ) and (2.3- 1 2). 
[Continuity of the entries of F(t)> etc., is sufficient to guarantee existence of a unique 
solution in a neighborhood of T.] Define the control law (not known to be optimal) 

«**(/ ) - -R- x (t)G'(t)P(t)x(t) 

and show that 

V(x(t 0 ), «(•)> t 0 ) = x'(t 0 )P(t 0 )x(t 0 ) + f T (u — u**)'R(u ~ it**) dx. 

J to 

Conclude that if P(t) exists for all te[t 0 , T], the optimal control is, in fact, w**. 

Problem 2.3-3. Find the optimal control for the system (with scalar u and *) 

x = u jt(t 0 ) given 

and with performance index 

V(x(t 0 ), «(•), t 0 ) = f (« 2 + x 2 ) dt -V x 2 (T). 

J to 

Problem 2.3-4. Find the optimal control for the system (with scalar u and x) 

x = x 4- u x(t 0 ) given 
and with performance index 

V(x(t 0 l «(•), t 0 ) = \ T (u 2 + 2x: 2 ) dt. 

J to 

Problem 2.3-5. Another version of the regulator problem imposes a further 
constraint — namely, that x(T) = 0 at the final time T. A class of performance indices 
implies intrinsic satisfaction of this constraint: 

V(x(to\ «(*), to) = f r (u'Ru + x'Qx) dt 

J to 

4- lim nx'(T)x(T). 

/|— >oo 

Show that if P satisfies the Riccati equation 

-P = PF + F'P - PGR-'G'P + Q, 
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and if exists, then P - * 1 satisfies a Riccati equation. Then show that P~ l (T ) = 0 
gives a local solution to the Riccati equation for P~\ which, on account of the con- 
tinuous dependence of solutions of differential equations on the boundary condition, 
defines the optimal performance index as x'(t)P(t, T)x(t) for t suitably close to T . 
Then show that this formula is valid for all t < T. 
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CHAPTER 




THE STANDARD REGULATOR 

PROBLEM— II 



3.1 THE INFINITE-TIME REGULATOR PROBLEM 



In this section, we lift the restriction imposed in the last chapter that the 
final time (the right-hand endpoint of the optimization interval), T, be finite. 
We thus have the following problem. 



Infinite-time regulator problem. Consider the system 



x = F(t)x{t) + G(t)u(t) x(t Q ) given (3.1-1) 

with the entries of F(t), G(t ) assumed continuous. Let the matrices Q(t ) 
and R(t) have continuous entries, be symmetric, and be nonnegative 
and positive definite, respectively. Define the performance index 



/• OO 

V(x{tol w(-), V) = (u'(t)R(t)u(t) + x’(t)Q(t)x{t)) dt (3.1-2) 

* 'o 



and the minimization problem as the task of finding an optimal control 
t > t 0 > minimizing V and the associated optimum performance 
index V*(x(t 0 \ t 0 ). 



It is not always possible to solve this problem as it is stated. To give 
some idea of the difficulty, consider the system 
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X 

m 

X 



1 

2_ 





*iOo) 

_* 2 <>o)- 



with performance index defined by R — [1] and 



It is readily established that 




1 0 ~ 

0 1 




Wo), &(•), 0 = f (w 2 + e 2t ) dt. 

J to 

In a sense, V is minimized by taking u — 0; but the resulting value of V is 
still infinite. 

With the finite-time problem, the optimal V is always finite; this may 
not be so in the infinite-time case. For the example given, it is clear that V 
becomes infinite for the following three reasons: 

1. The state jc 2 (/ 0 ) is uncontrollable. 

2. The uncontrollable part of the system trajectory is unstable (x x (/) 
~ c 0* 

3. The unstable part of the system trajectory is reflected in the system 
performance index ( e 2t is integrated). 

The difficulty would not have arisen if one of these reasons did not apply. 

It is intuitively clear that any regulator problem where one had a situation 
corresponding to that described by (1), (2), and (3) cannot have a finite optimal 
performance index. Therefore, to ensure that our problems are solvable, 
we shall make the following assumption. 



Assumption 3.1-1. System (3.1-1) is completely controllable for every time t. 
That is, given an arbitrary state x(t) at time t> there exists a control depending 
on x{t) and t , and a time t 2 depending on t, such that application of this control 
over the interval [ t , t 2 ] takes the state x(t) to the zero state at time t 2 . 

In the problems at the end of the section, we shall indicate a variant 
upon this assumption that will also ensure solvability of the infinite-time 
regulator problem. 

We now state the solution to the infinite-time regulator problem, under 
Assumption 3.1-1. 

Solution to the infinite-time regulator problem . Let P(t, T) be the solution 
of the equation 

-P =PF+ F'P - PGR~ l G'P + Q (3.1-3) 

with boundary condition P(T, T) == 0. Then lim P(t, T) = P(t) exists 
for all t and is a solution of (3.1-3). Moreover, x\t)P(t)x(t) is the optimal 
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performance index, when the initial time is t and the initial state is x{t). 
The optimal control at time t when the initial time is arbitrary is 
uniquelyt given by 

u*(0 = -R~ l {t)G\t)P(t)x(t) (3.1-4) 

(assuming t lies in the optimization interval). 

Evidently, we need to establish four separate results: (1) the existence of 
P(t); (2) the fact that it is a solution of (3.1-3); (3) the formula for the optimal 
performance index; and (4) the formula for the optimal control. The proof 
of (3) and (4) will be combined. 

Existence of P(t). Since (3.1-1) is completely controllable at time /, 
there exists for every x(t) a control w(«) and a time / 2 such that w(*) transfers 
x{t) to the zero state at time t 2 . Although w(«) is initially defined only on 
(A AL we extend the definition to [t, oo) by taking «(*) to be zero after t 2 . 
This ensures that the system will remain in the zero state after time t z . The 
notation V{x{t), w(-), t , T) will be used to denote the performance index 
resulting from initial state *(/) at time t , a control w(-), and a final time T y 
which is finite rather than infinite as in (3.1-2). Then P(t , T) exists for all T 
and / < T. Moreover, 

x'(t)P(t, T)x(t) = U T) 

<1 V(x(t), U[ t>T ), t , T) 

< V(x(t ), w (fj0o) , t , oo) 

< OO. 

Since .x(f) is arbitrary, it follows that the entries of P(t, T) are bounded inde- 
pendently of T. [Note that as T approaches infinity, if any entry of P(t , T) 
became unbounded, there would have to be a diagonal entry of P{t , T) which 
became unbounded, tending to infinity; consequently, for a suitable x(t), 
x'(t)P(t, T)x(t) would be unbounded.] 

Reference to (3.1-2), with T replacing +oo on the integral, and use of 
the nonnegative definite and positive definite character of Q and R , respec- 
tively, show that 

x\t)P(t, T 0 )x(t) < x'(t)P(t, T x )x{t) 

for any T x >T Q . The bound on P(t, T ) together with this mo noton ichy 
relation guarantees existence of the limit P(t). In more detail, the existence 

fAt least, u*(t ) is uniquely defined up to a set of measure zero, unless we insist on some 
property such as continuity. Henceforth, we shall omit reference to this qualification. 
(Those unfamiliar with measure theory may neglect this point.) 
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of the limit p a (t) will follow by taking x(f) = e iy for each i, and applying the 
well-known result of analysis that a bounded monotonically increasing func- 
tion possesses a limit. (Recall that e t is defined as a vector with zeros for all 
entries except the ith, where the entry is 1.) The existence of p u (t) will follow 
by observing that 2p u (t, T) = (e, + ej)'P(t, T)(e i + ej ) ~ p H (t , T) - Pjj (t, T), 
and that each term on the right possesses a limit as T approaches infinity. 



P(t) satisfies the Riccati equation (3.1-3). Denote the solution of (3.1-3) 
satisfying P(T) = A by P(t , T; A). [Then P(t, T), defined earlier, is P(t , T; 0).] 
Now a moment’s reflection shows that 



P(t, T; 0) = P(t, T^, P(T X , T; 0)) 
for t < T x < T, and thus 

P(t) = lim P(t, T; 0) - lim P(t, T x ; P(T U T; 0). 

T -* 00 T^°o 

For fixed time T Xi the solution P(t , T x ; A) of (3.1-3) depends continu- 
ously on A ; therefore, 

P(0=P(t,T 1 ;UmP(r l ,T; 0)) 

T-+ «, 

= P(t,T x ;P(T x )), 

which proves that P(t) is a solution of (3.1-3) defined for all /, 



Optimal performance index and control formulas. We show first that if 
the control defined by (3.1-4) is applied (where there is no assumption that 
this control is optimal), then 



V(x(t), w*(-), /, 00) = lim V(x(t% */*(•), /, T) = x'(t)P(t)x(t). 

T 



(3.1-5) 



Direct substitution of (3.1-4) into the performance index (3.1-2), with the 
initial time replaced by t and the final time by T, leads to 

V{x{t), «*(•), t, T) = x’(t)P(t)x(t) - x'(T)P(T)x(T) 

< x'{t)P{t)x{t), 



and, therefore. 



lim V(x(l), «*(•), 1, T) < x(t)P{t)x{t). 

P — ♦C-O 

Also, 



and, therefore. 



V(x(t), «*(.), T) > V*(x(t), t, T) 

> x\t)P{t, T)x(t ) 

lim V(x(t), «*(•), U T) > x\t)P(t)x(t). 

y— >oo 



The two inequalities for lim V(x(t), «*(•), /, T) then imply (3.1-5). Since 

X* — ►oo 
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u*(') has not yet been shown to be optimal, it follows that 

V*(x(t), I, oo) < F(x(0, «*(• ), t, oo). (3.1-6) 

We now show that the inequality sign is impossible. Suppose that strict 
inequality holds. Then there is a control u u different from w*, such that 

lim V(x{t), Wj , 7, T) = V*(x(t), 7, oo). 

T — *<x> 

Since, also, from (3.1-5), 

lim V*(x(t), t, T) = V(x(t), u*, t, oo), 

T —*°o 

it follows that strict inequality in (3.1-6) implies 

lim V (x(7), Mj, 7, F) < lim K*(x(7), 7, T). 

T T — >00 

This, in turn, requires for suitably large T 

V(x(t), u l9 7, T) < V*(x{t), 7, T). 

This is plainly impossible by the definition of the optimal performance index 
as the minimum over all possible indices. Consequently, we have established 
that x\t)P{t)x{t) is the optimal performance index for the infinite-time 
problem, and that —R~ l (t)G r (t)P(t)x(t) is the unique optimal control because 
it achieves this performance index, thereby completing the formal solution to 
the infinite-time regulator problem. 

It is of interest, for practical reasons, to determine whether time-invariant 
plants (3.1-J) will give rise to time-invariant linear control laws of the form 

u(t) = K f x(t). (3T-7) 

For finite-time optimization problems of the type considered in the last 
chapter, no choice of T , /?(•), and Q{ •) will yield a time-invariant control law 
when Fand G are constant, unless the matrix A takes on certain special values. 
[Problem 3.1-2 asks for this fact to be established.] For the infinite-time prob- 
lem, the case is a little different. Let us state an infinite-time problem that will 
be shown to yield a constant control law. In the next chapter, we shall con- 
sider a variation on this problem statement which also yields a constant (and 
linear) control law. 

Time-invariant regulator problem . Consider the system 

x = Fx + Gu x(7 0 ) given (3.1-8) 

where F and G are constant. Let the constant matrices Q and R be non- 
negative and positive definite, respectively. Define the performance 
index 

poo 

V(x(t 0 ), «(•), /„) = (u’Ru + x'Qx) dt (3.1-9) 

J t o 

and the minimization problem as the task of finding an optimal control 
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w*(*) minimizing V, together with the associated optimum performance 
index. 

To ensure solvability of the problem, it is necessary, as before, to make 
some additional restrictions. We shall require the following specialization of 
Assumption 3.1-1. 

Assumption 3.1-2. The system (3.1-8) is completely controllable. 

Since the system (3.1-8) is time invariant, there is no distinction between 
complete controllability at all times and complete controllability at some 
particular time. 

Solution to the time-invariant regulator problem . Let P(t, T ) be the solu- 
tion of the equation 

~P = PF + F'P - PGR~ l G'P + Q (3.1-3) 

with initial condition P(T \ T) — 0. Then lim P(t , T) — P exists and is 

« f — >OQ 

constant; also, P = lim P(t y T). Furthermore, P satisfies (3.1-3); that is, 

t->—oo 

PF + F’P- PGR' l G'P+ Q = 0 (3.1-10) 

and x\t)Px(t) is the optimal performance index when the initial time 
is t and the initial state is x(j). The optimal control at time t when the 
initial time is arbitrary is uniquely given by the constant control law 

u*(t) = -R-'G'Pxit). (3.1-11) 

Given the time-varying results, it is quite straightforward to establish 
the various claims just made. 

First, lim P(t , T) certainly exists. Now the plant is time invariant, and the 

T 

function under the integral sign of the performance index is not specifically 
time dependent. This means that the choice of the initial time is arbitrary — 
i.e., all initial times must give the same performance index, which is to say that 
P is constant. Likewise, because the initial time is arbitrary, 

P = lim P(t , T) = lim P( 0, T — t) = lim P( 0, T — t) — lim P(t , T). 

r-oo r-»— oo 

For the (time- varying) infinite-time problem, it was established that 
P(t) satisfies the Riccati equation (3.1-3). Consequently, the constant P 
here satisfies (3.1-3); since P is now zero, (3.1-10) follows. (It is sometimes said 
that P is a steady state solution of the Riccati equation.) 

The remainder of the claims are immediate consequences of the more 
general results applying for the time-varying problem. 

To illustrate the preceding concepts, we can consider the following 
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simple example. The prescribed system is 

x = x + u 

and the performance index is 

oo 

K(x(J 0 ), w(-), / 0 ) = (u l + x 2 ) dt. 

To find the optimal control, we solve 

-P = 2P - P 2 + 1 P{T, T ) - 0. 

It is readily found that 

Pit T) = ex P [ 2+/~2 (t P)] 

(a/TT + 1) + (V'2 — 1) exp [ — 2 a/ 2 ( t — P)] 

and 

lim P(t,T) = p = + 1. 

/ — > — oo 

(Observe, also, that 0 = 2P — P 2 + 1, as required. Indeed, this equation 
could have been used to determine P, assuming we knew which of the two 
solutions of the equation to choose.) The optimal control is 

w* = — (\fl + l)x 

(and thus the closed-loop system becomes x = ~*/~2x). 

A second example is provided by a voltage regulator problem, discussed 
in [1] and [2]. The open-loop plant consists of a cascade of single-order 
blocks, of transfer functions 



3 3 6 3.2 2.5 

OAs + 1 ’ 0.04s + 1 ’ 0.07 s + 1 ’ 2s + 1 ’ 5s + 1 ' 

In other words, the Laplace transform T(s) of the deviation from the correct 
output is related to the Laplace transform U(s) of the deviation from the 
reference input by 

WrA 3 3 6 3.2 2.5 t u \ 

K } OAs + 1 * 0.04s + 1 ‘ 0077+7 * 27+ I * 5s + 1 

State-space equations relating w(*) to X0 are readily found to be 




[Here, x l9 x 2> . . . , x s have physical significance; x 5 is the output of a block 
of transfer function 3(0. Is + 1) _1 with input u, x 4 the output of a block of 
transfer function 3(0.4s + l)' 1 with input x 5 , etc. Also, x { = y.] 
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As a first attempt at optimization, the performance index 

/• GO 

(x 2 + w 2 ) dt 

*1 ta 

is considered. (Of course, x\ = x'Qx, where is a matrix the only nonzero 
entry of which is unity in the 1-1 position.) 

Solution of the appropriate Riccati equation requires procedures dis- 
cussed subsequently in this book. The important quantity calculated is not 
so much the matrix P but the optimal gain vector. The optimal control law 
is, in fact, found to be 

w = —[0.9243 0.1711 0.0161 0.0392 0.2644]*. 

This problem also emphasizes the fact that the regulator theory is appli- 
cable to systems where the state is to be controlled around a nominal nonzero 
value, with a control varying around an associated nominal nonzero value. 



Problem 3.1 -1 . Consider the time-invariant problem, and suppose F and G 
have the form 



F = 



•f 1 1 


F t 2 ~ 






G - 


_o 


Pll - 




.0 _ 



so that the pair [F, G] is not completely controllable. [One way to see this is to 
examine the rank of [G FG ... ]; see Appendix B.] Show that if the eigenvalues 
of F 2 2 have negative real part, it is still possible to find a solution to the infinite- 
time regulator problem. Suppose that the solution J P(t) of the Riccati equation 
associated with the regulator problem is partitioned as 




Pm~ 
P 22- 



Show that Pi\ satisfies a Riccati equation, that P 12 satisfies a linear differential 
equation — assuming P x t is known — and that P {1 satisfies a linear differential equa- 
tion — assuming P 21 is known. (This problem is discussed in [3].) 



Problem 3.1 -2. Consider the finite-time regulator problem posed in the pre- 
vious section, where the linear system is time invariant. Show that if A — 0, there 
is no way, even permitting time-varying R(t) and Q(t), of achieving a constant con- 
trol law. Show that if R and Q are constant, then there is at least one particular 
A that will achieve a constant control law, and that this A is independent of 
the final time T. [Hint: For the second part of the problem, make use of Eq. 
(3.1-10).] 



Problem 3.1-3. Find the optimum performance index and optimal control 
for the system * = ax + u with performance index (w 2 + bx 2 ) dt , where a and 
b are constant and b is positive. Write down the equation of the closed-loop system, 
and show that the closed-loop system is always asymptotically stable. 
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3.2 STABILITY OF THE TIME-INVARIANT REGULATOR 

In this section, we shall be concerned with the stability of the closed- 
loop system formed when the control law resulting from an infinite-time 
performance index is implemented. Throughout, we shall assume a constant 
system that is completely controllable: 

x=Fx + Gu. (3.2-1) 

We shall also assume a performance index with constant nonnegative definite 
Q and constant positive definite R: 

/* OO 

V(x« 0 ), «(•), t 0 ) = ( u'Ru + x'Qx) dt. (3.2-2) 

J to 

We recall that the optimal performance index is x'(t 0 )Px(t 0 ), where P is the 
limiting solution of a Riccati equation ; also, P satisfies the algebraic equa- 
tion 

PF+ F'P — PGR- l G'P+ Q = 0. 

The optimal control is given by 

u* = -R-'G'Px, 

and, accordingly, the closed-loop system becomes 

x =(F — GR~ 1 G'P)x. (3.2-5) 

We ask the question: When is (3.2-5) an asymptotically stable system? 
Certainly, (3.2-5) is not always stable. Consider the example * - x + w, 
with V = J~ u 2 dt . Immediately, the optimal control is u ~ 0, and the 
closed-loop system is x = x, which is plainly unstable. In this instance, there 
are two factors contributing to the difficulty. 

1. The original open-loop system is unstable. 

2. The unstable trajectories do not contribute in any way to the perfor- 
mance index — in a sense, the unstable states are not observed by the 
performance index. 

Intuitively, one can see that if (1) and (2) were true in an arbitrary optimiza- 
tion problem, there would be grounds for supposing that the closed-loop 
system would be unstable. 

Accordingly, to ensure asymptotic stability of the closed-loop system, 
it is necessary to prevent the occurrence of (1) and (2). This motivates the 
introduction of the following assumption, which will be shown to guarantee 
stability of the closed-loop system (see also Problem 3.2-1 for an alternative 
assumption). 



(3.2-3) 

(3.2-4) 
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Assumption 3.2-1. The pair [ F , D] is completely observable , where D is any 
matrix such that DD' = Q. (Equivalently, the equation D'e Ft x 0 = 0 for all 
/ implies x 0 = 0, see Appendix B.) 

We can note at once that the question of whether Assumption (3.2-1) 
holds is determined by Q alone, and not by the particular factorization DD\ 
To see this, suppose D { and D 2 are such that D 1 D\ = D 2 D 2 = Q, with 
[F, D j] completely observable and [F, Z) 2 ] not completely observable. Then 
there exists a nonzero x 0 such that D' 2 e Ft x 0 = 0 for all t, and we have 

D' 2 e Ft x 0 = 0 for all t => x , 0 e F ' t D 2 D f 2 e Ft x Q = 0 for all t 

— => x t Qe Ft D i D\e Ft x Q = 0 for all t 

=> D\e Ft x 0 — 0 for all t 

which contradicts the complete observability of [F, /),]. Hence, either 
[F, Z)j] and [F, D 2 ] are completely observable simultaneously, or they are not 
completely observable simultaneously. 

Assumption 3.2-1 essentially ensures that all trajectories will show up in 
the x'Qx part of the integrand of the performance index. Since the perfor- 
mance index is known a priori to have a finite value, it is plausible that any 
potentially unstable trajectories will be stabilized by the application of the 
feedback control. 

The actual proof of asymptotic stability requires us first to note the fol- 
lowing result. 

Lemma. Consider the time-invariant regulator problem as defined by Eqs. 
(3.2-1) through (3.2-5) and the associated remarks. Then Assumption 3.2-1 is 
necessary and sufficient for P to be symmetric positive definite. 

Proof \ From the form of the performance index (3.2-2), it is clear that 
x'(t 0 )Px(t Q ) must be nonnegative for all x{t 0 ). Suppose for some nonzero x 0 
we have x' 0 Px 0 = 0, with Assumption 3.2-1 holding. Now the only way the 
integral in (3.2-2) will turn out to be zero is if the nonnegative integrand is 
always zero. This requires the optimal control to be zero for all t ; consequently, 
the system state at time t when the optimal control is applied is simply 
exp [F(/ — / 0 )]x 0 . Furthermore, 

_ Co o P co 

0 = x' 0 Px o = x'Qxdt = x'o exp [F'(t - t 0 )]DD' exp [F(t - t 0 )]x 0 dt. 

•7 to J to 

Now, Assumption 3.2-1 leads to a contradiction, because the preceding equa- 
tion implies D exp [F(t — tf)\xo — 0 for all /, and we have assumed jc 0 to be 
nonzero. 

To establish the converse, suppose there exists a nonzero x 0 such that 
D' exp [F(t — / 0 )]x 0 = 0 for all /. Apply the control u(t) = 0 for all t to the 
open-loop plant. It then follows that the associated performance index is zero. 
Since the optimal performance index is bounded below by zero, it follows that 
it, too, is zero — i.e., x' 0 Px Q = 0, which implies finally that P is singular. 
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Before applying the lemma, we recall two results in the theory of 
Lyapunov stability. The first is well-known, whereas the second is perhaps not 
so familiar but may be found in [4] and [5], 

Theorem A. Consider the time-invariant system x = f(x), with /( 0) = 0 and 
/(•) continuous. Suppose there is a scalar function V(x) that is positive definite, 
approaches infinity as ||x|| approaches infinity, and is differentiable. Suppose 
also that the derivative of V along system trajectories — namely, V = 
[<? Vjdx]'f(x) — is negative definite. Then the system is globally asymptotically 
stable. 

For the second theorem, we relax the negative definiteness assumption 
on V. A well-known result yields that if V is nonpositive definite, stability 
rather than asymptotic stability prevails. But by sharpening the constraint 
on V slightly, we have the following. 

Theorem B. Consider the time-invariant system x = f(x), with /( 0) = 0 and 
/(•) continuous. Suppose there is a scalar function V(x) that is positive definite, 
approaches infinity as \\x\\ approaches infinity, and is differentiable. Suppose 
also that the derivative of V along system trajectories — namely, V = 
[d V/dxYf(x) — is nonpositive definite, and not identically zero on [t u oo] for 
any t l9 save for a trajectory starting in the zero state. Then the system is globally 
asymptotically stable. 

The proof of asymptotic stability of the closed-loop system (3.2-5) 
is now straightforward. From (3.2-3), it follows that 

P(F — GR-'G'P) + (F - PGR~ l G')P = - g - PGR-'G'P . (3.2-6) 

Take V(x) — x'Px as a prospective Lyapunov function for the closed- 
loop system (3.2-5). Then the lemma ensures that V(x) is positive definite, 
whereas (3.2-6) implies that 

V(x) = -x'Qx - x'PGR-'G'Px. (3.2-7) 

Now, if Q is positive definite, as distinct from merely nonnegative definite, 
then V is negative definite, and thus we have asymptotic stability from Theo- 
rem A . 

Suppose now Q is nonnegative definite [of course, Assumption (3.2-1) 
remains in force]. Certainly V is nonpositive, by inspection. To conclude 
asymptotic stability using Theorem B, we must show that V is not identically 
zero along system trajectories starting from nonzero initial states. 

Suppose V is identically zero along a trajectory starting from a nonzero 
initial state x 0 . Then x' PGR~ l G' Px is identically zero; furthermore, 
— R~ l G f Px(t), which is the (optimal) control for the open-loop system, 
is also identically zero. Therefore, the trajectories of the closed-loop system 
are the same as those of the open-loop system and are given from x(t) = 
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exp [F(t — f 0 )]x 0 . Because the assumption V is identically zero also implies 

that x'Qx is identically zero, then x' 0 exp [F'(t — t 0 )]DD' exp [F(i — t 0 )]x Q 

must be identically zero. This contradicts Assumption 3.2-1 or the fact that 

0 

Xq ^ 0. Consequently, it is impossible to have V identically zero along a 
trajectory, other than that starting at the zero state. The asymptotic stability 
of (3.2-5) is thus established. 

The practical implications of the stability result should be clear. Nor- 
mally no one wants an unstable system; here, we have a procedure for guar- 
anteeing that an optimal system is bound to be stable, irrespective of the 
stability of the open-loop plant. The result is in pleasant contrast to some of 
the results and techniques of classical control, where frequently the main 
aim is to achieve stability and questions of optimality occupy an essentially 
secondary role in the design procedure. The contrast will actually be height- 
ened when we exhibit some of the additional virtues of the optimal regulator 
solution in later chapters. 

The interested reader may wonder to what extent the stability result 
applies in time-varying, infinite-time problems. Jn general, further assump- 
tions than merely Assumption 3.2-1 are needed, as discussed in Sec. 14.2; 
reference [6] should also be consulted. On the other hand, reference [7] gives 
in some detail the theory of the time-invariant problem. 

To illustrate these ideas, we recall briefly the two examples of the pre- 
vious section. Jn the first case, we had 



x = x + u K(*(/ 0 ), w(-), t 0 ) = (u 2 + x 2 ) dt. 

J to 

Obviously, the observability condition is satisfied. Therefore, it is no sur- 
prise that the closed-loop system, computed to be x = — \/~2x, is asymp- 
totically stable. 

For the voltage regulator example, we recall that the F matrix is 



~— 0.2 0.5 0 

0 -0.5 1.6 

0 0 -| 

0 0 0 

0 0 0 

and the weighting matrix Q is 



1 0 0 
0 0 0 
0 0 0 
0 0 0 
0 0 0 



0 0 - 

0 0 

T 0 

-0.25 7.5 

0 — 0.1 _ 



0 

0 

0 

0 

0 



0 ~ 
0 
0 
0 
0 
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A matrix D such that DD f = Q is given by D' = [1 0 0 0 0]. To check 

the observability condition, the easiest way, see Appendix B, is to examine the 
rank of [D F'D • • • (F') 4 D]. This matrix is readily checked to be trian- 

gular with nonzero elements all along the diagonal. Hence, it has rank 5. 
Consequently, the closed-loop system is known to be asymptotically stable. 

This could also be checked, with considerable effort, by evaluating the 
characteristic polynomial associated with the closed-loop system, and then 
checking via the Routh or other procedure that the roots of the polynomial 
possess negative real parts. There is little point in such a calculation here, 
however. 



Problem 3.2-1. With quantities as defined in Eqs. (3.2-1) through (3.2-5), 
suppose also that 



F = 



~F U 

0 



F 12“ 
F 22 — 



and 



D' 



[0 Z>i 2 ] with DD' 




(Then [F, D ] is not completely observable.) Show that the closed-loop system is 
asymptotically stable if all eigenvalues of F x x have negative real parts. 



Problem 3.2-2. The scalar version of (3.2-3) is a simple quadratic equation, 
which, in general, has more than one solution. Likewise, when (3.2-3) is a true 
matrix equation, there is generally more than one solution. Show that there is only 
one positive definite solution under Assumption 3.2-1. [This identifies the limiting 
solution of the Riccati equation uniquely as the positive definite solution of (3.2-3).) 
{Hint: Suppose there are two solutions, P x and P 2 > both positive definite. Define 
F t = F — GR~ 1 G'P i and show that both F, have eigenvalues with negative real 
parts. Prove that (F, — P 2 )Fj + F' 2 (P i — P 2 ) = 0 and use the result of [8] that the 
matrix equation AX + XB = 0 has the unique solution X = 0, provided X t [A] 
+ Xj[B] ^ 0 for any i and /; see also Appendix A for this result.) 

Problem 3.2-3. Can you state some explicit conditions under which the 
closed-loop system will fail to be asymptotically stable? 



Problem 3.2-4. Consider again Problem 3.1-3 of this chapter. Is there any 
relation between the exponential rate at which P(t , T) approaches P as t approaches 
minus infinity and the exponential rate of decay of a nonzero initial state in the 
closed-loop system? 



3.3 SUMMARY AND DISCUSSION OF THE REGULATOR 
PROBLEM RESULTS 



In this section, we first summarize all the important regulator problem 
results hitherto established. 
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Regulator problem and solution . Consider the system 

x = F(t)x + G(t)u x(t 0 ) given (3.3-1) 

with the entries of F and G assumed continuous. Let the matrices Q(t) 
and R(t ) have continuous entries, be symmetric, and be nonnegative and 
positive definite, respectively. Let A be a nonnegative definite matrix. 
Define the performance index 

V(x(t 0 ), «(•), t Q ) = f ( u(t)R(t)u(t ) + x'(t)Q(t)x{t)) dt 

J to 

+ x’(T)Ax(T) (3.3-2) 

where T is finite. Then the minimum value of the performance index is 

V*(x(to\ t 0 ) - x'(t 0 )P(t 0 , T)x(t 0 ) (3.3-3) 

where P(t , T) is the solution of the Riccati equation 

-P = PF + F'P - PGR-'G'P + Q (3.3-4) 

with boundary condition P(T , 7) = The matrix P(t, T) exists for all 
t < T. The associated optimal control is given by the linear feedback law 

«*(/) - - R~\t)G'(t)P(t , T)x(0. (3.3-5) 

Infinite-time regulator problem and solution . Suppose the preceding 
hypotheses all hold, save that ,4 — 0. Suppose also that the system 
(3.3-1) is completely controllable for all time. Then 

P(t ) - lim P(t , 7) (3.3-6) 

T’-.OO 

exists, and the minimum value of the performance index (3.3-2) with 
T replaced by infinity is x'(/ 0 )7(^ 0 )*(^o)- The matrix 7 satisfies the 
Riccati equation (3.3-4), and the optimal control law is 

u*(t) = ~R~ l (t)G\t)P(t)x(t). (3.3-7) 

Time-invariant regulator problem and solution . Suppose the hypotheses 
required for the infinite-time regulator problem and solution are present, 
and suppose also that the matrices F \ G , Q , and R are constant. Then 

P - lim />(/, T) - lim P(t, T) (3.3-8) 

T — «> /-»-oo 

is constant, and satisfies the algebraic equation 

PF + F’P - PGR~ l G'P + Q = 0. (3.3-9) 

The optimal control law is a time-invariant law 

u*(t) - -R-'G’Px{t). (3.3-10) 

Asymptotically stable time-invariant regulator problem and solution . 
Suppose the hypotheses required for the infinite-time regulator problem 
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and solution are present, and suppose also that the pair [F, D] is com- 
pletely observable where D is any matrix such that DD ' = Q. Then 
P is positive definite [and, by Problem (3.2-2), is the only solution of 
(3.3-9) with this property.] Moreover, the optimal closed-loop system 

x ~[F — GR~ { G'P]x (3.3-11) 

is asymptotically stable, and x Px is a Lyapunov function. 

Now that the solution to the regulator problem has been determined, 
we might ask if this solution is of interest for application to situations other 
than those in which everything is set up as in the formal statements just 
summarized. 

Frequently, the system states are not available for use in a controller 
input. In Chapters 8 and 9, we show that if the system is specified in the 
form 

x=^Fx + Gu y = H'x, (3.3-12) 

where only the input u and output y are available for controller inputs, a 
state estimator can be constructed with inputs u and y and output x, an esti- 
mate of the state x. Implementing the control law u = — R~ l G'Px is then 
a satisfactory alternative to implementing the control law u — — R~ l G'Px . 

So far in our discussions, we have assumed that the performance index 
matrices R and Q are specified. Since specification is usually up to the 
designer, a range of values for these matrices can be considered and by “trial 
and error” the most suitable values selected. This approach is often inefficient 
and leaves something to be desired. Of course, if we are only interested in 
stability, then we have the result that if the open-loop system jc = Fx is 
unstable, the closed-loop system x = (F — GR~ l G' P)x is always stable, irre- 
spective of the choice of Q and R within their prescribed limits. In other 
words, we have a general method for stabilizing multiple input linear systems 
(assuming that state estimators may be constructed). 

We are now led to ask whether linear regulators, optimal in the sense 
previously discussed, have desirable properties for the engineer other than 
simply “stability.” Or we might ask whether there are methods for selecting 
the index parameters Q and R so that desirable properties, such as good 
transient response and good sensitivity characteristics, are achieved. 

With these questions in mind, we now move on to the next section and 
chapter, which discuss various extensions of the regulator problem, and then 

to the following few chapters, which deal with further properties of regulator 
systems. 



Problem 3.3-1. State the physical significance of choosing Q = HH' and 
R = I (the unit matrix) in setting up a performance index for the system 
(3.3-12). 
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3.4 EXTENSIONS OF THE REGULATOR PROBLEM 

There are many avenues along which the regulator problem can be 
extended; a number of them will be explored in later chapters. We shall con- 
tent ourselves here by stating a few examples and indicating more completely 
one simple extension. The following are four examples of extensions to be 
more fully explored later. 

1. Time-invariant systems, with performance indices of the form 
j^ o e2at (u'R u + x'Qx) dt , where a, Q , and R are all constant. These 
will be shown to lead to a constant control law, and to a closed-loop 
system with improved stability properties. 

2. Performance indices of the form (u'Ru + x' Qx) dt. These lead 
to feedback laws that are no longer memoryless but dynamic. Regu- 
lators designed using this approach can accommodate slowly varying 
input disturbances. 

3. Performance indices of the form \ r tQ [u’Ru + (x — x)'Q(x — *)] dt , 
where x is a prescribed vector function. This is the servomechanism 
or tracking problem, where one desires the system states to follow 
a prescribed trajectory x. 

4. Performance indices of the form x'Qxdt . For this case, a con- 

straint on the magnitude of u is also included. The resultant optimal 
system operates in two modes. 

The approach used in developing the preceding extensions is to reduce by 
a suitable transformation the given problem to the standard regulator prob- 
lem. To illustrate this, we now consider a straightforward example of the 
technique. One application of this extension will be made when we consider (4). 

We consider the determination of an optimal control and associated 
optimal performance index for the system 

x — F(t)x + G(t)u x(t 0 ) given (3.4-1) 

when the performance index is 

V(x(t 0 ), w(.), t 0 ) = f [u\t)R(t)u{t) + 2x’(i)S(t)u(t) 

J to 

+ x'(t)Q(t)x(t)] dt (3.4-2) 

with R positive definite and the following constraint holding: 

Q-SR~'S'> 0 (3.4-3) 

(shorthand for Q - SR 1 S' is nonnegative definite). If desired, Tcan be in- 
finite, and F, G, Q, R, and S constant. 

To reduce this problem to one covered by the previous theory, we note 
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the following identity, obtained by completing the square: 

u'Ru + Ix’Su +x'Qx=(u + R~ l S'x)'R(u + R'S'x ) 



Making the definition 



+ x'(Q~ SR~ l S')x. 



w, = w + R'S'x, (3.4-4) 

the original system (3.4-1) becomes equivalent to 

x = (F — GR'S')x + Gw, (3.4-5) 

and the original performance index is equivalent to 

V(x (to), «,(•). to) = f T I"', * M i + *'«2 - SR- 'S')*] dt. (3.4-6) 

J to 

If u and Wj are related according to (3.4-4), the trajectories of the two systems 
(3.4-1) and (3.4-5) are the same [provided they start from the same initial 
state *(f 0 )]. Furthermore, the values taken by the two performance indices — 
viz., (3.4-2), which is associated with the system (3.4-1), and (3.4-6), which is 
associated with the system (3.4-5) — are also the same. Consequently, the 
following statements hold. 



1. The optimal controls u* and wf for the two optimization problems 
are related by wf = u* + R l S'x. 

2. The optimal performance indices for the two problems are the same. 

3. The closed-loop trajectories (when the optimal controls are imple- 
mented) are the same. 

Now the optimization problem associated with (3.4-5) and (3.4-6) is 
certainly solvable [and here we are using the nonnegativity constant (3.4-3)]. 
The optimal wf is given by 

uHO = -Rr l (t)G'(t)P(f, T)x(t ), 

where 



~P = P(F- GR-'S ') + C F ' - SR-'G')P 
- PGR-'G'P + Q~ SR' 1 S' 

with P(T, T ) = 0. The optimal index is x’(t 0 )P(t 0 , T)x(t 0 ). The optimal con- 
trol for the optimization problem associated with (3.4-1) and (3.4-2) is thus 

u*(t) = -R->(t)[ G '(t)P(t, T) + S\t)\x(t) 

and the optimal performance index is again x'(t 0 )P(t 0 , T)x(t 0 ). 



To consider the infinite-time problem [i.e., T in (3.4-2) is replaced by 
infinity], we make the following assumption. 



Assumption 3.4-1 . The system (3.4-1) is completely controllable at every time t. 
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To ensure existence of the limit as T approaches infinity of P(t, T), one 
evidently requires that the system (3.4-5) be completely controllable at every 
time t. [Then the optimization problem associated with (3.4-5) and (3.4-6) 
is solvable and yields a solution of the optimization problem associated with 
(3.4-1) and (3.4-2).] The complete controllability of (3.4-5) is an immediate 
consequence of the following lemma. 

Lemma. Suppose the system 

x = F(t)x + G(t)u (3.4-1) 

is completely controllable at some time t x . That is, with an arbitrary state Jc(^), 
there is associated a control «(•) and time t 2 such that (3.4-1) is transferred by 
w(') from the state *(/]) at time t x to the zero state at time t 2 . Then, with 
K(t) a matrix of continuous functions, the system 

x - [F(t) + G(t)K'(t)]x -|- G(t)u (3.4-7) 

[obtained from (3.4-1) by linear state-variable feedback] is completely control- 
lable at t x . 

Proof. With x(t x ) and */(•) as defined in the statement of the lemma, let x(r) 
be the resulting state of (3.4-1) at time /. Then 

i(0 = F(t)x(t) + G(t)u{t) 

= [F(t) + G(t)K'(t)]x(t) + G{t)[H(t) - K\t)x(t)l 

which is to say that if the control u(t) — K'(t)x(t) is applied to (3.4-7) with state 
x(?j) at time t u Eq. (3.4-7) follows the same trajectory as (3.4-1), and, in partic- 
ular, reaches the zero state at time / 2 - Equivalently, (3.4-7) is completely con- 
trollable at time and the lemma is established. 

The control law for the infinite-time problem will be constant if F, G, Q , 
R, and S are all constant, as may easily be seen. The closed-loop system will 
be asymptotically stable if the pair [Z 7 — GR ' l S\ D] is completely observable, 
where D is any matrix such that DD' — Q — SR 'S'. (If D were such that 
DD f = Q , note that complete observability of [F, D] or [F — GR~ l S\ D] 
would not necessarily imply asymptotic stability.) 



Problem 3.4-1 . Generalize the lemma of this section to nonlinear systems 
of the form x = f{x , u) with nonlinear feedback. 

Problem 3.4-2. Can you suggest a physical situation where the optimization 
problem associated with (3.4-1) and (3.4-2) would be appropriate? 

Problem 3.4-3. State and prove a result dual to that of the lemma concern- 
ing the complete observability of a class of systems. 
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CHAPTER 




THE REGULATOR WITH 
A PRESCRIBED DEGREE 

OF STABILITY 



4.1 QUALITATIVE STATEMENT OF THE REGULATOR 
PROBLEM WITH A PRESCRIBED DEGREE OF 
STABILITY 

Throughout this section, we shall normally restrict attention to time- 
invariant systems (F and G constant) of the form 

x = Fx + Gu. (4.1-1) 

Although the results presented in this chapter extend to time-varying sys- 
tems, they prove to be more meaningful for time-invariant systems. 

We shall also be interested in considering the effect of applying control 
laws of the form 

u ~ K'x (4.1-2) 

to (4.1-1), where K is a constant matrix. The associated closed-loop system 
is, of course, 

x — (F tF GK')x. (4.1-3) 

So far, we have described one procedure for the selection of control 
laws, this being based on the formulation of an infinite-time optimization 
problem. In fact, we recall from Sec. 3.3 that if (1) [F, G] is completely con- 
trollable, and (2) Q is symmetric nonnegative definite and R is symmetric 
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positive definite, where Q and R are both constant, then the problem of mini- 
mizing the performance index 

/* OO 

V(x{t Q ), «(•), t 0 ) = (« 'Ru + x’Qx) dt (4.1-4) 

J t Q 

is solved by using a feedback law of the form of (4.1-2). (For the moment, we 
shall not be concerned with the scheme for calculating K.) We recall, too, 
from Sec. 3.3 that if (3) [F, D] is completely observable, where D is any matrix 
such that DD r = Q, then the resulting closed-loop system is asymptotically 
stable. 

There are, of course, other procedures for determining control laws 
of the form (4.1-2), which might achieve goals other than the minimization 
of a performance index such as (4.1-4). One other goal is to seek to have all 
the eigenvalues of the closed-loop system (4.1-3) taking prescribed values. 
The task of choosing an appropriate K has been termed the pole-positioning 
problem. 

To solve the pole-positioning problem, the complete controllability 
of [F, G] is required. (For a proof of this result, see [1].) The task of comput- 
ing K in the single-input case is actually quite straightforward, if one converts 
F to companion matrix form and g to a vector containing all zeros except 
in the last position. (The complete controllability of [F, g] is actually suffi- 
cient to ensure the existence of a basis transformation in the state space 
taking F and g to the special form; see Appendix B.) Now if 

0 1 0 ■ • • 0 

0 0 1 0 

• • * ■ 

• • • • 

0 0 0 • - • 1 

(4 \ G 2 — Q 3 • ■ ’ — Cl 

a choice of k' = [£, k z • ■ • k n ) causes 

“0 1 0 0 
0 0 1 ... 0 

F + gk'= ' 

* 

0 0 0 ... 1 

a 2 k 2 o 3 -F k 3 — a n -f- k n 

and the eigenvalues of F + gk' become the roots of the equation 
+ (Pn ~ k n ) sn 1 + • ■ • + (g 2 — k 2 )s + (a l — /cj) = 0. 
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If these eigenvalues are prescribed, the determination of the feedback 
vector k , knowing a Y through a n , is immediate. For the multiple-input case, 
where G is no longer a vector but a matrix, the computational task is much 
harder but is nevertheless possible (see [2] through [6]). 

From the practical point of view, the essential problem may not be to 
fix precisely the eigenvalues of F + GK\ but rather to ensure that these eigen- 
values be within a certain region of the complex plane. Typical regions might 
be that sector of the left half-plane Re[s] < 0 bounded by straight lines 
extending from the origin and making angles ±0 with the negative real axis, 
or, again, they might be that part of the left half-plane to the left of 
= —a, for some a > 0. 

We shall be concerned in this chapter with achieving the latter restric- 
tion. Moreover, we shall attempt to achieve the restriction not by selecting 
K through some modification of the procedure used for solving the pole- 
positioning problem, but by posing a suitable version of the regulator prob- 
lem. Essentially what we are after is a solution of the regulator problem that 
gives a constant control law and that gives not merely an asymptotically 
stable closed-loop system, but one with a degree of stability of at least a. 
In other words, nonzero initial states of the closed-loop system (4.1-3) should 
decay at least as fast as e~ at . This is equivalent to requiring the eigenvalues 
of F + GK' to have real parts less than —a. 

As we shall illustrate, commencing in the next section, it proves appro- 
priate to replace the performance index (4.1-4) usually employed in the time- 
invariant regulator problem by a performance index 

/•oo 

V(x{t 0 ), m(-), t 0 ) — e 2at (u'Ru + x'Qx) dt (4.1-5) 

J fo 

where R and Q are as before — i.e., constant, symmetric, and respectively 
positive and nonnegative definite. The constant a is nonnegative (of course, 
a = 0 corresponds to the situation considered in Chapter 3). 

Since the integrand in (4.1-5) can be rewritten as uRu + x'Qx, where 
R = Re 2at , Q = Qe 2(Xt , the results summarized in Sec. 3.3 will guarantee 
that a linear feedback law will generate the optimal control for (4.1-5). That 
this law should be constant is not at all clear, but it will be proved in the next 
section. However, supposing that this is proven, we can give plausible reasons 
as to why the closed-loop system should have a degree of stability of at least 
a. It would be expected that the optimal value of (4.1-5) should be finite. 
Now, the integrand will be of the form e lctt x for some constant M 
when u is written as a linear constant function of x. For the integrand to 
approach zero as t approaches infinity, it is clearly sufficient, and probably 
necessary, for x(f) to decay faster than e~ at . This is equivalent to requiring 
the closed-loop system to have a degree of stability of at least a. The results 
of this chapter first appeared in [7]. 
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4.2 QUANTITATIVE STATEMENT AND SOLUTION OF THE 
REGULATOR PROBLEM WITH A PRESCRIBED 
DEGREE OF STABILITY 



The remarks of the previous section justify the formal statement of the 
modified regulator problem in the following manner. 

Modified regulator problem . Consider the system 

x == Fx + Gu x(t o) given (4.2-1) 

where Fand G are constant and the pair [F, (7] is completely controllable. 
Consider also the associated performance index 

poo 

K(x(/ 0 ), w(-), /„) = e 2 «'(u'Ru + x'Qx ) dt (4.2-2) 

where R and Q are constant, symmetric, and respectively positive definite 
and nonnegative definite. Let a be a nonnegative constant (which will 
turn out to be the minimum degree of stability of the closed-loop system). 
With D any matrix such that DD f = Q, let [F, D] be completely observ- 
able. Define the minimization problem as the task of finding the 
minimum value of the performance index (4.2-2) and the associated 
optimal control. 

The reason for the complete controllability condition is the same as 
explained in the last chapter; it ensures that the infinite, as distinct from 
finite, time problem has a solution. The observability condition will be needed 
to establish the constraint on the degree of stability of the closed-loop sys- 
tem. 

The strategy we adopt in solving this modified problem is to introduce 
transformations that convert the problem to an infinite-time regulator prob- 
lem of the type considered in the last chapter. Accordingly, we make the 
definitions 

x(t) = e*'x(t) (4.2-3) 

u(t) = e“u(t). (4.2-4) 

Just as x(-) and «(•) may be related [via Eq. (4.2-1)], so x(-) and w(-) may be 
related. Observe that 

^ = = * e “'*(0 + e“x(t) 

— ax + e^Fx + e at Gu 

= (F+ CLl)x + Gu. (4.2-5) 

Thus, given the relations (4.2-3) and (4.2-4), the system equation (4.2-1) 
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implies the system equation (4.2-5). The converse is clearly true, too. Corre- 
sponding initial conditions for the two systems (4.2-1) and (4.2-5) are given 
by setting t = in (4.2-3) — i.e., x(t 0 ) = e ar °x(t 0 ). 

The integrand in (4.2-2) may also be written in terms of u and x: 

e lat (u'Ru + x'Qx) = uRu + x'Qx. 

Consequently, we may associate with the system (4.2-5) the performance 
index 



f 00 

V{x{t 0 \ u(-), r 0 ) = (d'Jta + x'Qx)dt. (4.2-6) 

*o 

Moreover, there is a strong connection between the minimization problem 
associated with the equation pair (4.2-1), (4.2-2), and the pair (4.2-5), (4.2-6). 
Suppose u*(t) is the value of the optimal control at time t for the first prob- 
lem, and that x(t) is the resulting value of the state at time t when the initial 
state is x(/ 0 ). Then the value of the optimal control at time t for the second 
problem is w*(/) = and the resulting value of the state at time t 

is given by x(r) = e at x(t), provided x(/ 0 ) = e ar °x(/ 0 ). Moreover, the minimum 
performance index is the same for each problem. 

Moreover, if the optimal control for the second problem can be expressed 
in feedback form as 

w*(r) = k(x(t), t ), (4.2-7) 

then the optimal control for the first problem may also be expressed in feed- 
back form; thus, 

w*(0 = e“ a 'w*(/) — e~ at k(e*‘x(t), t). (4.2-8) 

[We know that the control law (4.2-7) should be a linear one; and, indeed, 
we shall shortly note the specific law; the point to observe here is that a 
feedback control law for the second problem readily yields & feedback control 
law for the first problem, irrespective of the notion of linearity.] 

Our temporary task is now to study the system (4,2-5), repeated for 
convenience as 

i = (F + a/)x + Gu x(t 0 ) given (4.2-5) 

and to select a control w*(*) that minimizes the performance index 

^ r°° 

V(x(t 0 ), «(•), t 0 ) = (u'Ru + x'Qx) dt (4.2-6) 

J to 

where R and Q have the constraints imposed at the beginning of the section. 

As we discussed in the last chapter, this minimization problem may 
not have a solution without additional constraints of stability or controlla- 
bility. One constraint that will guarantee existence of an optimal control is, 
however, a requirement that [F + a/, G] be completely controllable. As 
will be shown, this is implied by the restriction that [F\ G] is completely con - 
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* 

trollable , which was imposed in our original statement of the modified regula- 
tor problem. To see this, we first need to observe a property of the complete 
controllability concept, derivable from the definition given earlier. This 
property, also given in Appendix B, is that [F, G ] is completely controllable 
if and only if the equation w'e Ft G — 0 for all t where w is a constant vector 
implies w = 0. 

The complete controllability of [F + a/, (7] follows from that of [F, (7] 
(and vice versa) by observing the equivalence of the following four statements. 

1 . [F, G] is completely controllable. 

2. For constant w and all ( , w f e Ft G ~ 0 implies w = 0. 

3. For constant w and all f, w'e at e Ft G = w'e (F+aI)t G = 0 implies w — 0. 

4. [F + a/, G] is completely controllable. 

Given this complete controllability constraint, we can define a solution 
to the preceding minimization problem with the most minor of modifications 
to the material of Sec. 3.3. Let P(t , T) be the solution at time t of the equation 

—P - P(F + a/) + (F' + a/)F - PGR~ l GP + Q (4.2-9) 
with boundary condition P(T, T ) = 0. Define 

P = lim P(t, T), (4.2-10) 

which is a constant matrix, satisfying the steady state version of (4.2-9): 

P(F + «/) + (F' + <x/)P - PGR~ l G'P +0=0. (4.2-1 1) 

Then the optimal control becomes 

u*(t) = -R- l G’Px{t). (4.2-12) 

It is interesting to know whether the application of the control law 
(4.2-12) to the open-loop system (4.2-5) results in an asymptotically stable 
closed-loop system. We recall from the results of the last chapter that a suffi- 
cient condition ensuring this asymptotic stability is that [F + a/, D] should 
be completely observable, where D is any matrix such that DD' = Q. Now, 
just as the complete controllability of [F, G\ implies the complete control- 
lability of [F + a/, G], so by duality, the complete observability of [F, D ] 
implies the complete observability of [F + txl, D). Since the complete observ- 
ability of [F, D] was assumed in our original statement of the regulator 
problem with degree of stability constraint, it follows that [F + a/, D ] is 
completely observable and that the closed-loop system 

i = (F + a/ - GR-'G'P)x (4.2-13) 

is asymptotically stable. 

We can now apply these results to the original optimization problem. 
Recall that we need to demonstrate, first, that the optimal control law is 
a constant, linear feedback law, and second, that the degree of stability of the 
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closed-loop system is at least a. Furthermore, we need to find the minimum 
value of (4.2-2). 

Equations (4.2-7) and (4.2-8) show us that 

u*(t) = -e-* t R’ 1 G'Pe* t x(t) = -R 'G'Px(t). (4.2-14) 

This is the desired constant control law; note that it has the same structure 
as the control law of (4.2-12). 

To demonstrate the degree of stability, we have from (4.2-3) that x(t) 
= e~ at x(t). Since the closed-loop system (4.2-13) has been proved asymptot- 
ically stable, we know that x(t) approaches zero as t approaches infinity, 
and, consequently, that x(t) approaches zero at least as fast as e~ ** when 
t approaches infinity. 

The minimum value achieved by (4.2-2) is the same as the minimum value 
achieved by (4.2-6). As was shown in the previous chapter, the optimal 
performance index for (4.2-6) is expressible in terms of P as x'(t 0 )Px(t 0 ). 
Consequently, the minimum value achieved by (4.2-2) is x'(t 0 )e~ 2ato Px(t 0 ). 
Let us now summarize the results in terms of the notation used in this 
chapter. 

Solution of the regulator problem with prescribed degree of stability . The 
optimal performance index for the modified regulator problem stated 
at the start of this section is x\t 0 )e~ 2cito Px{t {s ), where P is defined as the 
limiting solution of the Riccati equation (4.2-9) with boundary condition 
P(T , T) = 0. The matrix Palso satisfies the algebraic equation (4.2-11). 
The associated optimal control is given by the constant linear feedback 
law (4.2-14), and the closed-loop system has degree of stability of at least 
a. 



One might well ask if it is possible to construct a performance index with 
a equal to zero such that the control law resulting from the minimization is 
the same as that obtained from the preceding problem when a is nonzero. 
The answer is yes (see Problem 4.2-2). In other words, there are sets of pairs 
of matrices R and Q such that the associated regulator problem with zero 
a leads to a closed-loop system with degree of stability a. However, it does 
not appear possible to give an explicit formula for writing down these matrices 
without first solving a regulator problem with a nonzero. 

By way of example, consider an idealized angular position control 
system where the position of the rotation shaft is controlled by the torque 
applied, with no friction in the system. The equation of motion is 

J6 - T 

where 8 is the angular position, T is the applied torque, and J is the moment 
of inertia of the rotating parts. In state-space form, this becomes 
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X = 



0 r 

0 0 



X + 



0 

a 



\u 



where x, — 9, x 2 — 6, and a — TjJ. As a performance index guaranteeing 
a degree of stability a = 1, we choose 

/» CO 

e 2 '(« 2 + Jcf) dt. 

J 0 

The appropriate algebraic equation satisfied by P is 



1 1 
0 I 



+ 



1 0 
1 1 



P-P 



0 O' 
0 a 2 






1 0 
0 0 



= 0. 



One-possible way to solve this equation is to write down simultaneous equa- 
tions for the entries of P; thus, 

2 Pi i — Pha 2 + 1=0 
At +2 Pi 2 — Pi iP zl a 2 = 0 

2p.a + 2 Pn — Pha 1 = 0. 



These equations lead to 



P " ~ a 2 

1 



1 



2 + 2+1 + a 2 + -=-(2 + 2 a / T + _ o 2 ) 3/2 



A 2 — ^ 2 [1 + VI -ha 2 + V2 + 2+ 1 + a 2 ] 

Pii = ^[2 + V2 + 2+ 1 + a 2 ]. 

The optimal control law is 

«* = -g'+x 

-■^-(1 + VI + a 2 + V 2 + 2V1 + a 2 ) 

JJ"(2 + V 2 + 2+1 + c 2 ) 

This is implementable with proportional plus derivative (in this 
or angular velocity) feedback. The closed-loop system equation 

x = 0 1 

L-(l + VT+~V + VT+2+1 —) -(2 + V2T2VI + V). 

for which the characteristic polynomial is 

+ (2 + ^2 +2V1 + a 2 )s + (1 + vT+^l + V2 + 2 VI + a 2 ). 

It is readily checked that the roots of this polynomial are complex for all a; 



case, tacho, 
is 
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therefore, the real part of the closed-loop poles is 

-1 - 2 + 2 VI +a 2 < -1. 

Thus, the requisite degree of stability is achieved. 



Problem 4.2-1 . Consider the system (with constant F and G) 

x = Fx + Gu x(to) given 
and the associated performance index 

Poo 

e l *‘[(u'uy + (jc 'Qx)*]dt 

J to 

where Q is a constant nonnegative definite matrix. Find a related linear system and 
performance index where the integrand in the performance index is not specifically 
dependent on time (although, of course, it depends on u and x). Show that if an 
optimal feedback law exists for this related system and performance index, it is a 
constant law, and that from it an optimal feedback law may be determined for the 
original system and performance index. 

Problem 4.2-2. Consider the system 

x = Fx + Gu x(/ 0 ) given 

where Fand G are constant and [F, G] is completely controllable. Show that asso- 
ciated with any performance index of the form 

r* oq 

| e 2ctt (u'Ru + x'Qx) dt, 

J to 

where R is constant and positive definite, Q is constant and nonnegative definite, 
and a is positive, there is a performance index 

C°° S\ 

(u'Ru + x'Qx) dt y 
J to 

where Q is constant and nonnegative definite, such that the optimal controls asso- 
ciated with minimizing these indices are the same. [Hint: Define Q using the solution 
of the first minimization problem.] 



Problem 4.2-3. Consider the system 



with performance index 



x = ax + u 




+ bx l ) dt 



where a, h, and a are constants, with b and a nonnegative. Examine the eigenvalue 
of the closed-loop system matrix obtained by implementing an optimal feedback 
control, and indicate graphically the variation of this eigenvalue as one or more of 
a , b, and a vary. 



Problem 4.2-4. Consider the modified regulator problem as stated at the 
beginning of the section, and let P be defined as in Eqs. (4.2-9) through (4.2-11). 
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Show that V(x) = x'(t)Px(t) is a Lyapunov function for the closed-loop system with 
the property that VjV < —2a. (This constitutes another proof of the degree of 
stability property.) 

Problem 4.2-5. Suppose that you are given a linear time-invariant system with 
a feedback law K a derived from the minimization of a performance index 

Poo 

e 2ott (u'u + x'x) dt. 

J to 

Suppose also that experiments are performed to determine the transient response 
of the system for the following three cases. 

1. K* ~ K 0 — i.e., a is chosen as zero. 

2. Ka. = K %x — i.e., a is chosen as a “large,” positive constant. 

3. Ka = Ko 4- (K ai - AToXl - e~'). 

Give sketches of the possible transient responses. 

Problem 4.2-6. If F and G are constant matrices and F is n x n , it is known 
that [F, G\ is completely controllable if and only if the matrix [G FG • * • F n ~ { G] 
has rank n. Prove that complete controllability of [F, G] implies complete control- 
lability of [F 4- a/, G] by showing that if the matrix [G FG • • • F n ~ l G\ has rank 
/?, so, too, does the matrix [G (F -f a/) G • • • (F + a/)" -1 G]. 

Problem 4.2-7. Imagine two optimization problems of the type considered 
in this chapter with the same F, G, Q , and R but with two different a — viz., a* 
and a 2 , with a 2 > a 2 . Show that P ai — F* 2 is positive definite. 



4.3 EXTENSIONS OF THE PRECEDING RESULTS 



We now ask if there are positive functions f(t) other than e xt with the 
property that the minimization of 

poo 

v(x(t o), u(-), t a ) = f(t)(u'Ru + x'Qx) dt, (4.3-1) 

J to 

given that 

x = Fx + Gu x(t 0 ) given (4.3-2) 

leads to a linear constant control law. (Here, the usual constraints on F, G, 
Q, and F, including constancy, are assumed to apply.) Reference [8] suggests 
that maybe f(t) = t k could lead to a constant control law. 

We show here that essentially the only possible f(t) are those we have 
already considered— i.e., those of the form c a '~ but with the constraint that 

a should simply be real (rather than nonnegative, as required earlier in this 
chapter). 

By writing the integrand of (4.3-1) as u’[f(t)R]u + x [f(t)Q\x, it is evident 
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that the associated optimal control is 

«*(0 = -f- l (t)R- l G'P(t)x(t) (4.3-3) 

where P(-) is a solution of 

-p = PF+ F'P ~ f-'(t)PGR : G'P +f(t)Q. (4.3-4) 

Now (4.3-3) is to be a constant control law. Hence, the matrix P(t)G must 
be of the form f(t)M, where M is some constant matrix. Premultiply (4.3-4) 
by G' and postmultiply by G to obtain 

— G'Mf = [M'FG + G’F’M - G'MR 'M'G + G'QG\f (4.3-5) 

Suppose G'P(t)G = G'M is nonzero. Then there is at least one entry — 
say, the i-j entry — that is nonzero. Equating the i-j entries on both sides of 
(4.3-5), we obtain 

/ = */ 

for some constant a. Thus, the claim that/ has the form e %t is established for 
the case when G'P(t)G 0 for some /. Now suppose G’P(t)G = 0. Then, 
since P(t) is nonnegative definite, it follows that P(t)G = 0, and thus the opti- 
mal control is identically zero. In this clearly exceptional case, it is actually 
possible to tolerate /(/) differing from e at . Thus, we have the trivial example 
of 



poo 

V(x(t 0 ), u(-), t 0 ) = f(t)u’Ru dt, 

J t 0 

which for arbitrary positive /(/) and arbitrary F, G, and R has the constant 
optimal feedback law u ^ 0. 

Since the preceding two sections have restricted attention to the case 
of nonnegative a, it might well be asked why this restriction was made. The 
answer is straightforward : Constant control laws certainly result from negative 
a, but the resulting closed-loop systems may be unstable, although no state 
grows faster than e~ al . (Note that e~ at is a growing exponential, because a is 
negative.) 

An open problem for which a solution would be of some interest (if 
one exists) is to set up a regulator problem for an arbitrary system (4.3-2) 
such that the eigenvalues of the closed-loop system matrix F— GR~ l G'P 
possess a prescribed relative stability — i.e., the eigenvalues have negative 
real part — and if written as a + jco (<r, co real, j ~ *f— 1), the constraint 
\co\!\o\<k for some prescribed constant k is satisfied. 

A second open problem is to set up a regulator problem such that the 
closed-loop system matrix possesses a dominant pair of eigenvalues (all 
but one pair of eigenvalues should have a large negative real part). Partial 
solutions to this problem will be presented in Chapter 5. 
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PART III 



PROPERTIES AND APPLICATION 
OF THE OPTIMAL REGULATOR 
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\ 




CHAPTER 



PROPERTIES OF REGULATOR 
SYSTEMS WITH A CLASSICAL 
CONTROL INTERPRETATION 



5.1 THE REGULATOR FROM AN ENGINEERING 
VIEWPOINT 



We have earlier intimated a desire to point out what might be termed the 
“engineering significance’’ of the regulator. Until now, we have exposed a 
mathematical theory for obtaining feedback laws for linear systems. These 
feedback laws minimize performance indices that reflect the costs of control 
and of having a nonzero state. In this sense, they may have engineering sig- 
nificance. Furthermore, we have indicated in some detail for time-invariant 
systems a technique whereby the closed-loop system will be asymptotically 
stable, and will even possess a prescribed degree of stability. This, too, has 
obvious engineering significance. 

Again, there is engineering significance in the fact that, in distinction to 
most classical design procedures, the techniques are applicable to multiple- 
input systems, and to time-varying systems. (We have tended to avoid dis- 
cussion of the latter because of the additional complexity required in, for 
example, assumptions guaranteeing stability of the closed-loop system. How- 
ever, virtually all the results presented hitherto and those to follow are appli- 
cable in some way to this class of system.) 

But there still remains a number of unanswered questions concerning the 
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engineering significance of the results. For example, we might well wonder to 
what extent it is reasonable to think in terms of state feedback when the states 
of a system are not directly measurable. All the preceding, and most of the 
following, theory is built upon the assumption that the system states are 
available; quite clearly, if this theory is to be justified, we shall have to indi- 
cate some technique for dealing with a situation where no direct measurement 
is possible. We shall discuss such techniques in a subsequent chapter. Mean- 
while, we shall continue with the assumption that the system states are avail- 
able. 

In classical control, the notions of gain margin and phase margin play 
an important role. Thus, engineering system specifications will often place 
lower bounds on these quantities, since it has been found, essentially empiri- 
cally, that if these quantities are too small system performance will be 
degraded in some way. For example, if for a system with a small amount of 
time delay a controller is designed neglecting the time delay, and if the phase 
margin of the closed loop is small, there may well be oscillations in the actual 
closed loop. The natural question now arises as to what may be said about 
the gain margin and phase margin (if these quantities can, in fact, be defined) 
of an optimal regulator. 

Of course, at first glance, there can be no parallel between the dynamic 
feedback of the output of a system, as occurs in classical control, and the 
memoryless feedback of states, as in the optimal regulator. But both schemes 
have associated with them a closed loop. Figure 5.1-1 shows the classical 




Fig. 5.1-1 Classical feedback arrangement with dynamic control- 
ler driven by system output. 

feedback arrangement for a system with transfer function h'(sl — F)~ l g, 
where the output is fed back through a dynamic controller with transfer func- 
tion 0(s). Figure 5.1-2 shows a system with transfer function h'(sl - F)~'g 
but with memoryless state-variable feedback. Here a closed loop is formed; 
however, it does not include the output of the open-loop system, merely the 
states. This closed loop is shown in Fig. 5.1-3. 

Now it is clear how to give interpretations of the classical variety to the 
optimal feedback system. The optimal feedback system is like a classical 
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Fig. 5.1-2 Modern feedback arrangement with memoryless con- 
troller driven by system states. 




Fig. 5.1-3 The closed-loop part of a feedback system using 
modern feedback arrangement. 



situation where unity negative feedback is applied around a (single-input, 
single-output) system with transfer function — k\sl — F)~'g. Thus, the gain 
margin of the optimal regulator may be determined from a Nyquist, or some 
other, plot of W(jco) = —k'(jcoI— F)~ l g in the usual manner. 

We may recall the attention given in classical control design procedures 
to the question of obtaining satisfactory transient response. Thus, to obtain 
for a second-order system a fast response to a step input, without excessive 
overshoot, it is suggested that the poles of the closed-loop system should have 
a damping ratio of about 0.7. For a higher order system, the same sort of 
response can be achieved if two dominant poles of 0.7 damping ratio are used. 
We shall discuss how such effects can also be achieved using an optimal 
regulator; the key idea revolves around appropriate selection of the weight- 
ing matrices ( Q and R) appearing in the performance index definition. 

A common design procedure for systems containing a nonlinearity is 
to replace the nonlinearity by an equivalent linear element, and to design and 
analyze with this replacement. One then needs to know to what extent the 
tiue system performance will vary from the approximating system perfor- 
mance. As will be seen, a number of results involving the regulator can be 
obtained, giving comparative information of the sort wanted. 

We shall also consider the question of incorporating relay control in 
otherwise optimal systems. There may often be physical and economic 
advantages in using relay rather than continuous control, provided stability 
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problems can be overcome; accordingly, it is useful to ask whether any 
general remarks may be made concerning the introduction of relays into 
optimal regulators. 

A further question of engineering significance arises when we seek to 
discover how well an optimal regulator will perform with variations in the 
parameters of the forward part of the closed-loop system. One of the common 
aims of classical control (and particularly that specialization of classical 
control, feedback amplifier design) is to insert feedback so that the input- 
output performance of the closed-loop system becomes less sensitive to varia- 
tions in the forward part of the system. In other words, one seeks to desen- 
sitize the performance to certain parameter variations. 

The quantitative discussion of many of the ideas just touched upon 
depends on the application of one of several basic formulas, which are derived 
in the next section. Then we pass on to the real meat of the regulator ideas in 
this and the next two chapters. 



5.2 SOME FUNDAMENTAL FORMULAS 

To fix ideas for the remainder of this chapter, we shall restrict attention 
to closed-loop systems that are completely controllable, time invariant, and 
asymptotically stable. Thus, we shall take as our fundamental open-loop sys- 
tem 

x = Fx + Gu (5.2-1) 

with [F, G] completely controllable. As the performance index, we take 

oo 

Y(x(t 0 ), »(•), t 0 ) = e 2 "(u'Ru + x'Qx) dt (5.2-2) 

T t 0 

with the usual constraints on Q and R, including that [F, D] be completely 
observable for any D such that DD' = Q. At times, a will be zero. Since we 
shall be considering different values of a, we define P a , rather than P, as the 
unique positive definite solution of 

PJF + a/) + (F' + « I)P, - P^GR~ i G'P t + 2=0. (5.2-3) 

The optimal control law is 

u = K' x x = -R'G’P.x. (5.2-4) 

Using the notation A 0 = A a | (a =0), we shall prove first that 
[/ - R' ,2 K' 0 (-sI - F)-'GR-‘ 2 ]'[/ _ R>'2K' 0 (sI - FY'GR - '«] 

= /- R~ [1 G'(—sl - F’)~ l Q(sI - F) , GR~ ,n . (5.2-5) 

This is an identity that has appeared in a number of places (often merely in 
scalar form) — e.g., [1], [2], [3]. From (5.2-3), it follows that 

P 0 {sl -F) + (-si - F’)Po + K q RK'o = 2 - 
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Multiplying on the left by R~ x 2 G'(—sI ~ F') \ and on the right by 
(si- F)- L GR- l/ \ yields 

R ^G'i-sI-F'y'PoGR ' 1 + R~ 1/2 G'P q (sI- Fy x GR~ uz 

+ R~ x ;i G'(-sI- F')~ l K Q RK' 0 (sI - Fy l GR~ l/2 

= R~ h2 G’(—sI - F') x Q(s I - F) X GR x/2 . 

By adding I to each side, and observing that P Q GR~ UZ — — K 0 R X/Z , Eq. 
(5.2-5) follows. 

Let us now examine some variants of (5.2-5). With s = jco , Eq. (5.2-5) 
becomes 

[/- R X2 K f 0 (-jcoI - F)-'GR- l 2 ]'[I - R l/z K' Q (jcoI - F) l GR x/2 ] 

= J + R l z G'(-jcoI - F’y x Q(j(oI - Fy i GR ~ 1/2 . (5.2-6) 

With * denoting the complex conjugate, the left-hand side is Hermitian, 
whereas the right-hand side is of the form I + B'*(ja>)QB(ja>) for a matrix 
B. If we adopt the notation C, > C 2 for arbitrary Hermitian matrices C x and 
C 2 to indicate that C, — C 2 is nonnegative, and if we use the fact that B'*QB 
> 0, Eq. (5.2-6) implies that 

[/ — R U2 K f 0 (—jcoI — Fy i GR~ 1/2 ]'[I — R ]/2 K' 0 (jcoI — F)' X GR~ UZ ] > /. 

(5.2-7) 

For a single-input system (5.2-1), there is no loss of generality in taking R 
as unity. Equations (5.2-5), (5.2-6), and (5.2-7) then become 

[1 - k> 0 (-sl- F)-'g][l - k’ a (sI~Fy'g] 

= 1 +g’(-sr- F)-'Q(sI- F) ~'g, (5.2-8) 

or 

1 1 - Kijcol - F)-'g\> = 1 + g’(~jcoI - F')-'Q(jcoJ - F)~'g. 

(5.2-9) 

Therefore, we have that 

\1 - k' 0 (jcol - Fy'g\>\. (5.2-10) 

Problem 5.2-1 asks for the establishment of the result that the equality sign 
in (5.2-7) and (5.2-10) can hold only for isolated values of co. 

Relations (5.2-5) through (5.2-10) are all specialized to the case a = 0. 
For the case a ^ 0, there are effectively two ways of developing the corre- 
sponding relations. The first relies on observing, with the aid of (5.2-3), that 
P » satisfies the same equation as P 0 , save that F + #/ replaces F. Consequently, 
Eqs. (5.2-5) through (5.2-10), derived from (5.2-3), will be supplanted by equa- 
tions where F + a/ replaces F, and K t replaces K 0 . For example, (5.2-9) 
yields 

|1 - k'oijcol - F - aiy'gf 

= 1 + g'(-jcol — F' — al)Q(jcoI -F- a/)-'g, (5.2-11) 
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and, evidently, one could also regard this as coming from (5.2-9) by replacing 
k 0 by k a and ±ja> by ±jco — a (leaving F invariant). 

The second way of deriving relations corresponding to (5.2-5) through 
(5.2-10) for the case when a is nonzero is to observe, again with the aid of 
(5.2-3), that P a satisfies the same equation as P 09 save that Q + 2a P a replaces 
Q. Then Eq. (5.2-9), for example, is replaced by 

1 1 - KUcoi ~ ry i g\ z - 1 + g'(~jcoi - F'y i Q(jo>i - Fy'g 

+ 2ccg'(—jcoI — F'y i P a (jcoI — Fy i g. 

(5.2-12) 

The other equations in the set (5.2-5) through (5.2-10) will also yield new 
relations by simply replacing K 0 by K a and Q by Q + 2a P a . 

Problem 5.2-1 . Show that inequalities (5.2-7) and (5,2-10) become equalities 
only for isolated values of co. 

Problem 5.2-2. Show that (5.2-9) implies 

1 -11 Fk' 0 (jcoI-F-gk’ 0 y'g\> 

— g( jcol — F' — kog'y'QUcal -F- gk' 0 y'g. 

State two equivalent equations applying for a^O. 

Problem 5.2-3. Derive an equation analogous to (5.2-8) when u = k’ Q x is 
the optimal control for the system x = Fx + gu with a performance index 
(u 2 F2 x'S u + x'Qx) dt , and when Q — SS' is nonnegative definite symmetric. 

Problem 5.2-4. Assume that for a specific value of a — say, a = a — the 
matrix P ~ has been calculated as the positive definite solution of (5.2-3). As an appro- 
ximate technique for computing P a for values of a near a, one could set P a ~ 
Pa (a a)(dP a /dcc) \ (a — a). Show that dPJd a satisfies a linear algebraic 
matrix equation. 



5.3 GAIN MARGIN, PHASE MARGIN AND 
TIME-DELAY TOLERANCE 



In this section, we shall restrict attention to the regulator with a scalar 
input, and we shall examine certain properties of the closed-loop scheme of 
Fig. 5.3-1, which is a redrawn version of Fig. 5.1-3 (discussed earlier in this 
chapter), with the gain k replaced by Xr a . We shall be especially interested in 
the gain margin, phase margin, and time-delay tolerance of the scheme. 

We recall that the gain margin of a closed-loop system is the amount 
by which the loop gain can be increased until the system becomes unstable. 
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Fig. 5.3-1 Closed-loop optimal scheme with degree of stability 
a drawn as a unity negative feedback system. 

If the loop gain can be increased without bound — i.e., instability is not 
encountered, no matter how large the loop gain becomes — then the closed- 
loop system is said to possess an infinite gain margin. 

Of course, no real system has infinite gain margin. Such parasitic effects 
as stray capacitance, time delay, etc., will always prevent infinite gain margin 
from being a physical reality. Some mathematical models of systems may, 
however, have an infinite gain margin. Clearly, if these models are accurate 
representations of the physical picture — save, perhaps, for their representa- 
tion of parasitic effects — it could validly be concluded that the physical sys- 
tem had a very large gain margin. 

We shall now show that the optimally designed regulator possesses the 
infinite gain margin property, by noting a characteristic feature of the Nyquist 
diagram of the open-loop gain of the regulator. The scheme of Fig. 5.3-1 is 
arranged to have unity negative feedback, so that we may apply the Nyquist 
diagram ideas immediately. The associated Nyquist plot is a curve on the 
Argand diagram, or complex plane, obtained from the complex values of 

— k' a (jcoI — F)~ l g as co varies through the real numbers from minus to plus 
infinity. Now the Nyquist plot of —k' x (jcoI — F)~ } g is constrained to avoid 
a certain region of the complex plane, because Eq. (5.2-12) of the last section 
yields that 

|1 -K(jG)I-Fy>g\> 1, (5.3-1) 

which is to say that the distance of any point on the Nyquist plot from the 
point — 1 +y'0 is at least unity. In other words, the plot of ~k f a (jcol — F)~ 
avoids a circle of unit radius centered at —1 -f- JO. 

See Fig. 5.3-2 for examples of various plots. (The transfer functions are 
irrelevant.) The arrow marks the direction of increasing co. Note that the 
plots end at the origin, which is always the case when the open-loop transfer 
function, expressed as a numerator polynomial divided by a denominator 
polynomial, has the numerator degree less than the denominator degree. 

There is yet a further constraint on the Nyquist plot, which is caused by 
the fact that the closed-loop system is known to be asymptotically stable. 
This restricts the number of counterclockwise encirclements of the point 

— 1 + jO by the Nyquist plot to being precisely the number of poles of the 
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Fig. 5.3-2 Nyquist plots of -k'jjcol - avoiding a unit 

critical disc center (-1, 0). Points are at unity distance from the 
origin. 



transfer function k a (sl F ) 1 g lying in > 0. We understand that if a 

pole lies on ite(>] = 0, the Nyquist diagram is constructed by making a small 
semicircular indentation into the region Re[s] < 0 around this pole, and 
plotting the complex numbers — k\sl — F)~ l g as s moves around this semi- 
circular contour. (For a discussion and proof of this basic stability result, 
see, for example, [4].) 
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Fig . 5.3-3 Nyquist plot for which open-loop transfer function has 
two poles in Refy] > 0, and closed-loop is stable. 

Thus, if —k’fsl — F)~ l g has no poles in > 0, diagrams such as 
those in Fig. 5.3-2 may be obtained. Fig. 5.3-3 illustrates a case where 
—k'fsl — F)~ l g has two poles in Re[>] > 0. 

The key observation we now make is that the number of encirclements 
of the point — 1 +y*0 is the same as the number of encirclements of any other 
point inside the circle of unit radius and center — 1 + j0. The best way to see 
this seems to be by visual inspection; a few experiments will quickly show 
that if the preceding remarks were not true, the Nyquist diagram would have 
to enter the circle from which it has been excluded. 

It is known that the closed-loop system with gain multiplied by a con- 
stant factor p will continue to be asymptotically stable if the Nyquist dia- 
gram of —fik'aijcol — F)~ l g encircles —1 + y'O in a counterclockwise direc- 
tion a number of times equal to the number of poles of —fik'fsl — F)~ l g 
lying in /te[s] > 0. Equivalently, asymptotic stability will follow if the Nyquist 
diagram of —k' a (jcDl— F) l g encircles — (1/jS) + y’O this number of times. 
But our previous observation precisely guarantees this for a range of /?. 
The points — (1//?) +y'0 for all real p > \ lie inside the critical circle, and 
thus are encircled counterclockwise the same number of times as the point 
— 1 -f yO. As we have argued, this number is the same as the number of poles 
of - k'fsl - F)~ l g in Re[s] > 0. Consequently, with asymptotic stability 
following for all real p > we have established the infinite gain margin 
property. 

Let us now turn to consideration of the phase margin property. First, 
we recall the definition of phase margin. It is the amount of negative phase 
shift that must be introduced (without gain increase) to make that part of 
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the Nyquist plot corresponding to co > 0 pass through the —1 +j0 point. 
For example, consider the three plots of Fig. 5.3-2; points A at unit distance 
from the origin on the co > 0 part of the plot have been marked. The negative 
phase shift that will need to be introduced in the first and second case is 
about 80°, and that in the third case about 280°. Thus, 80° is approximately 
the phase margin in the first and second case, and 280° in the third. 

We shall now show that the phase margin of an optimal regulator is 
always at least 60°. The phase margin is determined from that point or those 
points on the co > 0 part of the Nyquist plot which are at unit distance from 
the origin. Since the Nyquist plot of an optimal regulator must avoid the 
circle with center — 1 + JO and unity radius, the points at unit distance from 
the origin and lying on the Nyquist plot of an optimal regulator are restricted 
to lying on the shaded part of the circle of unit radius and center the origin, 
shown in Fig. 5.3-4. The smallest angle through which one of the allowable 

* 



Fig. 5.3-4 Shaded points denote per- 
missible points on Nyquist plot of 
optimal regulator at unit distance from 
origin. 

points could move in a clockwise direction to reach — 1 + jO is 60°, corre- 
sponding to the point N of Fig. 5.3-4. Any other point in the allowed set of 
points (those outside the circle of center —1 + y0, and unity radius, but at 
unit distance from the origin) must move through more than 60° to reach 

— 1 +j 0. The angle through which a point such as N must move to reach 

— 1 +y0 is precisely the phase margin. Consequently, the lower bound of 
60° is established. 

Let us now consider the effect of varying a away from zero. Reference 
to Eq. (5.2-11) shows that 

1 1 - Kijcol — F — al)-'g\ = 1 1 - k'Jijco - a)/ - F]->g| > l. 

(5.3-2) 

Consequently, if a modified Nyquist plot is made of - «)/ - F]-'g 

rather than of —k'^jcol — F)~ l g, this modified plot will also avoid the circle 
of center —1 +j0 and radius 1. Then it will follow that if a gain /? > \ is 
inserted in the closed loop, the degree of stability a of the closed-loop system 
will be retained for all real /?. If a negative phase shift of- up to 60° is intro- 
duced, a degree of stability a will also still be retained. 





Sec. 5.3 



GAIN MARGIN, PHASE MARGIN AND TIME-DELAY TOLERANCE 75 



Yet another interpretation of the case of nonzero a follows by com- 
paring Eqs. (5.2-9) and (5.2-12) of Sec. 5.2, repeated here for convenience: 

1 1 - k f 0 (jcol - F) _1 g| 2 - 1 + g\~jcol - F'Y'QUtol - F)' l g 

and 

1 1 - K(joiI - F)-'g| 2 = 1 + g'(-jcoI - F‘y'Q(jcoI - F)-'g 

+ 2ag’(-jo)l - F’Y'PSjml - F) 'g, 

whence 

1 1 - K(jcoI - F) J g| 2 = |1 - k f Q (jcDl - F) _1 g| 2 

+ 2a g’i-jco! - F’Y'PJij&I - FY'g. 

(5.3-3) 

The second term on the right side is nonnegative, being of the form 
2ccb ( *(jco)P a b(jco) for a vector b. In fact, one can show that b is never zero, 
and so 

1 1 - K(jcoI - F)~'g\ > 1 1 - k’oijcol - FY'g\. (5.3-4) 

This equation says that any two points on the Nyquist plots of 
—k'aijcol — F) _1 gand ~k f Q (jorl — FY l g corresponding to the same value of 
or are such that the point on the first plot is further from — 1 + jO than the 
point on the second plot. In loose terms, the whole plot of — k' a (jcoI — F)~ ] g 
is further from — 1 + JO than the plot of —k’^jcol — FY l g. This does not, 
however, imply that the phase margin for the former is greater than that 
for the latter. Problem 5,3-1 asks for a proof of this somewhat surprising 
result. 

Problem 5.3-2 asks for a proof of the result that if > a 2 > 0, the 
Nyquist plot of — k! ai {j(x)l — F)" 1 g is further, in the preceding sense, from 
— 1 + jO than the plot of ~k f a2 (jcoJ — F)~ l g. 

We now turn to a discussion of the tolerance of time delay in the closed 
loop. Accordingly, we shall consider the scheme of Figure 5.3-5, where T 
is a certain time delay. (The delay block could equally well be anywhere else 
in the loop, from the point of view of the following discussion.) We shall 
be concerned with the stability of the closed-loop system. 




Fig. 5.3-5 Optimal regulator with time delay T inserted. 
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The effect of the time delay is to insert a frequency-dependent nega- 
tive phase shift into the open-loop transfer function. Thus, instead 
of —k'aijcol — Fy 1 g being the open-loop transfer function, it will be 
—k f a (jo)I — F)~ l ge~ ju>T . This has the same magnitude as — k'j^jcol — F)~ x g y 
but a negative phase shift of coT radians. 

It is straightforward to derive allowable values for the time delay which 
do not cause instability. Considering first the a = 0 case, suppose the transfer 
function —k' Q {j(j)l — F) x g has unity gain at the frequencies a> 15 a > 2 , . . . , c o r 
with 0 < co l < a> 2 < • • ■ < a> r , and let the amount of negative phase 
shift that would bring each of these unity gain points to the —1 + j0 point 
b e > 02> * * • > respectively. Of course, 0,. > nj 3 for all /. Then, if a time 
delay T is inserted, so long as cofT < 0. or T < 0 / /a> / for all z, stability will 
prevail. In particular, if T < n/3co r , stability is assured. 

For the case of a nonzero, since the modified Nyquist plot of 
—K[(jto — a)/ — F\~ l g has the same properties as the Nyquist plot of 
—k' 0 (jcol — F)~ l g, we see (with obvious definition of co^ . . . , co r and 
0] , . . . , that T < fr/cot for all /, and, in particular, T < 7ij3o) r will ensure 
maintenance of the degree of stability a. Greater time delays can be tolerated, 
but the degree of stability will be reduced. 

The introduction of time delay will destroy the infinite gain margin 
property. To see this, observe that as co approaches infinity, the phase shift 
introduced — viz., coT radians — becomes infinitely great for any nonzero T. 
In particular, one can be assured that the Nyquist plot of —k' a (j<x>I — F)~ l g 
will be rotated for suitably large co such that the rotated plot crosses the real 
axis just to the left of the origin. (In fact, the plot will cross the axis infinitely 
often.) If for a given T, the leftmost point of the real axis that is crossed is 
— (1//0 +y0, then the gain margin of the closed loop with time delay is 
20 log 10 db. 

Of course, the introduction of a multiplying factor, phase shift, or time 
delay in the loop will destroy the optimality of the original system. But the 
important point to note is that the optimality of the original system allows 
the introduction of these various modifications while maintaining the required 
degree of stability. Optimality may not, in any specific situation, be of direct 
engineering significance, but stability is; therefore, optimality becomes of 
engineering significance indirectly. 

Problem 5.3-1. Show with suitable sketches that the relation 
1 1 ~ kaijcol — F)~'g\ > 1 1 — ko(jcol — does not imply that the phase 

margin associated with the transfer function — k' a (jcoI — F)~ l g is greater than that 
associated with the transfer function —k^ijcol — F)~ ] g. 

Problem 5.3-2. Suppose a x > a 2 > 0. Show that | 1 - k' ai (jCOl - F)~'g | 

> 1 1 — k' ai (j(oi — F)~ l g |. [Hint: Use the result of Problem 4.2-7 of Chapter 4 that 
F ai — F at is positive definite.] 
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Problem 5.3-3. Suggest ways of extending the preceding material to multiple- 
input systems. 

Problem 5.3-4. Let u = k f 0 x be the optimal control for the system x = Fx 
+ gu with performance index (w 2 -f 2 x'Su + x Qx) dt, where it is assumed that 
Q — SS' is nonnegative definite symmetric. Show that it is not generally possible to 
conclude results of the sort described in this section. 



5.4 POSITIONING OF THE CLOSED-LOOP SYSTEM 
POLES 



In the last chapter, we pointed out that frequently a feedback law may 
be required such that the closed-loop system has certain desired poles. These 
poles may be specified so that, for example, a desired transient response may 
be obtained or a certain bandwidth may be achieved by the closed-loop 
system. Frequently, such specifications may call for some of the closed-loop 
poles to be dominant, with the implication that the real part of the other poles 
should be negative and large in magnitude. 

In this section, we shall present an optimal design procedure that will 
achieve approximately the aim of fixing the dominant closed-loop poles. 
The end result of the procedure is a set of dominant closed-loop poles that 
are close to the desired dominant poles, with the remaining nondominant 
poles possessing a large negative real part. The system specifications that 
generate a requirement for dominant poles will, in general, not require the 
exact realization of these dominant poles. Therefore, the design procedure, 
although approximate, is still generally useful in terms of meeting the pole 
position constraints. At the same time , an optimal system results , with all its 
attendant advantages . Chapter 7, Sec. 7.4 will discuss further techniques for 
pole positioning in optimal systems. 

A characteristic of the design procedure is that not all closed-loop 
poles may be dominant. Since the procedure allows specification in advance 
of only the dominant poles, the ability to specify all poles is lacking. What 
allows us to specify that the system is optimal is the lack of dominancy of 
precisely that pole (or those poles) which may not be specified. As remarked 
in the last chapter, if optimality is not desired, we can always find a linear 
feedback law for a linear system such that all closed-loop poles have desired 
values. Thus, to retain optimality, we are simply sacrificing the right to specify 
certain (in fact, the unimportant) pole positions. 

The advantages attendant in having optimality of the closed-loop system 
have already been partly indicated, but they will become clearer in the next 
two chapters. For the moment, the reader will still have to accept the fact 
that optimality has more to offer than a first glance would indicate. 
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As preliminary material associated with the design procedure we shall 
also indicate the following: 

1. A computational procedure (not involving the Riccati equation) 
for obtaining the optimal control law for a single-input time invariant 
system. 

2. A general property of the closed-loop system poles. (This property 
will be exploited in defining the design procedure.) 

Of course, (1) is of interest in its own right. Following discussion of (1) and 
(2), we shall develop the design procedure, first for single-input systems and 
then for multiple-input systems. 

To begin with, we shall restrict consideration to single-input systems 

x = Fx + gu (5.4-1) 

where F is an n x n matrix, and the pair [F, g] is completely controllable. 
We shall first derive a computational procedure , independent of use of the 
Riccati equation, for minimizing a performance index of the type 

p oo 

V(x(t 0 ), «(•), t 0 ) = (w 2 + x'Qx) dt (5.4-2) 

J to 

with Q = DD' for some D such that the pair [F, D] is completely observable. 

Under the stated controllability and observability conditions, we know 
that there is an optimal closed-loop law u = k'x, such that the closed-loop 
system is asymptotically stable and that the following equation [derived from 
the fundamental Eq. (5.2-8) of the earlier section], is satisfied: 

[1 _ *'(-*/- F)-'*][l - k'(sI-F)-'g] 

- 1 + g\-sl - FT l Q'(sI - F)~ x g. (5.4-3) 

Let y/(s) denote the characteristic polynomial of F— i.e., det (si — F). Define 
the polynomial p(s) via 

\-k\sI-F)-'g = M (5.4-4) 

and the polynomial q(s) by 

- n " e(s ' - F >~'e = .(-?$,(,) ■ < 5 -«) 

When j is set equal to infinity in this equation, the left-hand side is clearly 
zero. Hence, q(s) has degree less than twice that of y/(s), which has degree n. 
On other hand, p(s) has the same degree as y/(s), as may be seen by setting s 
equal to infinity in (5.4-4). Also, we see from (5.4-4) and the fact that yj(s) 
is monic that p(s) is also monic — i.e., its highest order term is s 

The polynomial p(s) has another important interpretation, which we 
may now derive. Observe that 
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[1 - k'(sl - F)-‘*][l + k’(sl - F - gk')-'g] 

= 1 - k\sl — F)~'g + k\sl -F- gk’Y'g 

- k\sl - F)-'[(sl - F) -(si- F- gk')](sl - F - gk'y'g 

= 1 . 



Thus, inversion of (5.4-4) gives 



and, accordingly, 



y'O) = i 

P(s) 



+ k'(sl — F — gk'y { g. 



p{s) — det (si — F — gk r ). 



(5.4-6) 



We recall that the closed-loop optimal system is x — (F + gk')x. Therefore, 
the zeros of p(s) are the poles of the closed-loop system transfer function. 
In particular, since the closed-loop system is asymptotically stable, we note 
that the zeros of p(s) must all have negative real parts. 

Equations (5.4-3), (5.4-4), and (5.4-5) combine to yield 



pf^Ypjs) = , , q(s) 
y/(-s)i/Y(s) y/(-s)\f/(s) 

which may also be written 

p(-s)p(s) = v(-s)y/(s) + q(s). 



(5.4-7) 

(5.4-8) 



It is this equation that may be used as a basis for determining the optimal 
control law /c, when F , g , and Q are known : All quantities on the right side are 
known, and thus p(—s)p(s) is known. The requirement that p(s) possess zeros 
all with negative real parts and be monic, then specifies p(s) uniquely from 
p(—s)p(s). Finally, Eq. (5.4-4) yields k. A summary of the method for deter- 
mining k follows. 



1. Form the polynomial y/(s) = det (si — F) and the polynomial 
q(s) =zg'(—sl — F'Y l Q(sI — Fy { gy/(s)y/(~ s). Notice that q(s) = 
q(-s). 

2. Construct the right-hand side of Eq. (5.4-8), and factor the resulting 
polynomial. This polynomial is even, and thus has the property that 
if a is a root, so is ~o . 

3. Select those roots a f of the polynomial having negative real parts, 
and construct II(s — er f -), which is a monic polynomial with roots 
these a t . 

4. Because the zeros of p(s ) must have negative real parts, and because 
P( s ) is bionic, it follows that p(s ) is uniquely determined by p(s) 

= n ^(i 1 — cT;). [Note also that the constraints on p(s ) guarantee that 

no o i in step 3 will have zero real part.] 
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5. Construct k so that 1 - k'(sl - FT 1 # = p(s)ly/(s). (This is a simple 
task requiring no more than the solving of linear equations which 
determine k uniquely.) 

Let us give an example of the preceding procedure. We take 




Then y/(s) = det (si — F) = s 2 + 1, and q(s) = 3. [Notice that q(s) is even, 
as required.] Consequently, p(—s)p(s) = (s 2 + 1)(^ 2 + 1) + 3 = s A + 2s 1 
+ 4. Now observe the factorization (s 4, + 2s 1 + 4) = (s 2 — \J~ls + 2) 
(j 2 + VTs + 2), from which it is clear that p(s) = s 2 + v'Tj + 2. The 
zeros of the polynomial p(s) are the closed-loop system poles. Meanwhile, 
1 - k f (sl — F)~ l g = (5 2 + VTs + 2)/(s 2 + 1) = 1 + (VT^ + l)/t* 2 + l). 
It is easy to check that 

and, consequently, k f = [— 1 — a/T]. 

We stress that this particular computation procedure is only applicable 
to time-invariant systems (in contrast to the approach using the Riccati 
equation), and could be extended but with difficulty to multiple-input systems. 
Moreover, the procedure involves the potentially difficult step of factoring 
a polynomial [the right side of (5.4-8)] and extracting out the roots with nega- 
tive real parts. Nevertheless, the procedure may be valuable for computational 
purposes, provided the system order is not high. It has additional theoretical 
advantages, which we shall now illuminate. 

We modify the original performance index (5.4-2), and replace it by 

poo 

V(x(t 0 ), «(•), t 0 ) = (w 2 + px'Qx ) dt (5.4-9) 

J t a 

where p is a positive constant. We shall examine the effect on the closed-loop 
poles of making p very large. Our conclusions concerning the variation will 
then enable us to present the design procedure to which we referred at the 
beginning of the section. 

Equation (5.4-8) will be replaced by 

p(-s)p(s) = y/{~s)y/(s) + pq(s). (5.4-10) 

The poles of the closed-loop system— i.e., the zeros ofp(s)— are given by those 
zeros of the polynomial \p{— s)ip(s) + pq(s) having negative real parts, or, 
equivalently, lying in the left half-plane Rc[s\ < 0. For the moment, let us 
disregard the problem of separating out those particular zeros of y{— s)y/{s) 
+ pq{s) that lie in the left half-plane, and instead concentrate on examining 
the behavior of all the zeros as p approaches infinity. 
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It is immediately clear that any zero of y/(~s)y/(s) + pq(s) that remains 
finite as p approaches infinity must tend toward a zero of q(s). Now q(s ) 
has degree 2 ri, which is less than the degree 2 n of y/(— s)y/(s) + pq(s) [see 
the remark immediately following (5.4-5)]. Therefore, 2 ri zeros only of 
y/(—s)y/(s) + pq{s) will remain finite as p approaches infinity, and the remain- 
ing 2 (n — ri) must tend to infinity. Let us consider the precise manner in 
which they do so. Clearly, for very large values of s, the polynomial y/(~s)y/(s) 
is approximated by its highest order term— viz., (— 1)V"; also, for very large 
values of the polynomial pq(s) is approximated by its highest order term, 
which is of the form (—1 Ypas ln ' for some constant a. Now (5.4-5) implies 
that q(joy) > 0 for all real co , which, in turn, implies that a is positive. Con- 
sequently, for large /?, those zeros of y/(~s)y/(s) + pq(s) that tend to infinity 
must approximately satisfy the equation 

(— 1 ) n s 2n + (— Xypas 2n ' = 0 

with <x a certain positive constant. Since the zero roots of this equation are 
irrelevant, the equation may be replaced by 

= (—l) (n - n ' +l) paL. (5.4-11) 

In summary, then, as p approaches infinity, 2 ri zeros of y/(—s)y/(s) + pq(s) 
approach the 2ri zeros of q(s), whereas the remaining 2 (n ~ ri) zeros approach 
the zeros of Eq. (5.4-11). 

The zeros of p(s ), or the closed-loop system poles, are the particular 
zeros of y/(— s)y/(s) + pq(s) having negative real parts. Therefore, as p 
approaches infinity, ri of the closed-loop system poles approach those zeros 
of q(s) with negative real part, whereas (n — ri) approach those roots of 
(5.4-11) with negative real part. 

The latter zeros lie in a pattern on a circle of radius (p(x) l/2(n ~ n, \ which 
network theory terms a Butterworth configuration [1]. The phase angles are 
given in the following table. 



n - n! = 1 


+ 

OO 

O 

o 


n — ri ~ 2 


±135° 


n - ri = 3 


±120°, +180° 


n — ri = 4 
etc. 


±112.5°, ±157.5° 



We shall now discuss the application of these ideas to a design procedure. 
Suppose we are given the wth-order plant (5.4-1), and a list of ri < n desired 
dominant poles z u . . . , z n >. We shall describe the setting up of an optimal 
regulator problem such that the associated optimal closed-loop system has 
dominant poles approximating the desired ones. 
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n 

We first form the polynomial m(s) = II (.? — z,). As before, y/(s) = 

I = 1 

det ( 5 / — F ). Then an n vector d may be defined by 

dXsI-n-'g^V&y (5.4-12) 

and using d , we define the matrix (9 of the performance index (5.4-9) as Q 
= dd'. We assume [F, d] is completely observable. (See problem 5.4-10 for 
the contrary case.) 

Retaining the definition (5.4-5) of the polynomial q(s ), we can check 
simply that q(s) is precisely m(~s)m(s): 

= g\-sI-F')- t dd'(sl-F)- l g 
_ m(—s ) m(s) 

~vi~s)vts) 

We recall also that for suitably large p in the performance index (5.4-9), ri 
of the optimal closed-loop system poles are approximated by the negative 
real part zeros of q(s). But since q(s ) is m(—s)m(s) and m(s) has negative real 
part zeros, it follows that for suitably large p the optimal closed-loop domi- 
nant poles are approximated by the zeros m(s ), which are precisely the desired 
dominant closed-loop poles. 

Having fixed the matrix Q of the performance index (5.4-9) as Q = dd\ 
we now ask what the value of p should be. The choice of p is governed by the 
need to ensure that the n f zeros of m(s) are truly dominant poles of the closed- 
loop system — i.e., that the remaining poles of the closed-loop system should 
have magnitude at least 10 times the magnitude of any of the dominant poles. 
Now the magnitude of the nondominant poles is approximately 
(as the constant a of our earlier remarks is now unity). Therefore, p might 
be chosen sufficiently large to guarantee that 

p l/1( "- n,) >10\z t \ ; = 1, (5.4.13) 

In summary, the design procedure is as follows: 

1. With n dominant poles, form a polynomial m(s) of degree n’ , with 

highest order term s"', and such that m{s) has the desired dominant 
poles as its zeros. 

2. Define the vector d by Eq. (5.4-12). 

3. Choose Q = dd' and p so that Eq. (5.4-13) is satisfied. The perfor- 
mance index (5.4-9) will be minimized by an optimal control that 
determines a closed-loop system possessing the approximately desired 
dominant poles. 

4. Determine the feedback law either by the Riccati equation approach 

or by the procedure given earlier in this section based on factoriza- 
tion of (5.4-10). 




Sep. 5.4 



POSITIONING OF THE CLOSED-LOOP SYSTEM POLES 83 



To illustrate the procedure, we shall consider two examples. First, sup- 
pose 




roi 




and we wish to have a dominant pole of the closed-loop system at — 1 + jO. 
(Note that we cannot specify two poles of the closed-loop system.) The 
polynomial m(s) — s + 1 has the single root s = — 1 ; now form m(s)jy/(s) = 
(s + l)l(s 2 + 1), and observe that d' = [1 1] ensures that d\sl — F)' l g = 

m(s)jy/(s). We have also that n — ri = 1, and Eq. (5.4-13) yields that p > 100. 
We shall therefore take p = 100, and minimize the performance index 
§?A u2 + 100(x, + x i ) 2 )dt. The closed-loop system poles are the negative 
real part zeros of 



t//(~s)tg(s) + pm(~s)m(s) = j 4 + 2s 2 + 1 — 100s 2 + 100 



= j 4 - 9&s 2 + 100 



+ 20^ 2 + 100 - 118s 2 

= (s 2 + vTT8 s + 10 )(s 2 - vTT85 + 10). 

Accordingly, the closed-loop poles are the zeros of s 1 + vTTBj + 10, which 
are computed to be —1.01 and —9.84. The dominant pole is then correct to 
1 %. The feedback vector k (generating the optimal control according to 
u — k'x) is given by 



1 - k\sl 

from which we derive 



F)~ l g 



s 2 + +/TWs + io 
s 2 + l 



k' = [-9 -vlT8]. 

For a second example , we consider the sixth-order system defined by 



■"0 


1 


0 


0 


0 


(T 




~ 0 ~ 


0 


0 


1 


0 


0 


0 




0 


0 


0 


0 


1 


0 


0 




0 


0 


0 


0 


0 


1 


0 


g = 


0 


0 


0 


0 


0 


0 


1 




0 


J) 


0 


0 


+1 


+1 


- 1 _ 







We aim to have dominant poles at s — — 1, s = — 1 The open-loop 
poles are given by the zeros of det (si — F) = y/(s ), which may be computed 
to be s 3 (s + l) 2 (s — 1) = s 6 + s 5 — .s 4 — s 3 . In terms of the earlier notation, 
the polynomial m(s) is (s + l)(s 2 + 2s + 2) = s 3 + 3s 2 + 4s + 2. By 
forming m(s)ly/(s) = (s 3 + 3s 2 + 4 s + 2 )/(s 6 + s s — s 4 — s 3 ), we can check 
that d' = [2 4 3 1 0 0] ensures that d’(sl — F)~ x g = m(s)j\f/(s). We 

have also that n — n' =3, and so Eq. (5.4-13) yields that p> 2 6 x 10 6 . 
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However, it turns out that smaller values of p will give reasonably acceptable 
results. Thus, with p = 10 5 , the closed-loop poles are the negative real part 
zeros of s 12 — 3s 10 + 3s 8 — s 6 + 10 5 (— s 6 + s* — 4s 2 + 4); a computer 
evaluation of the zeros of this polynomial leads to the closed-loop poles 
— 1.00, —1.00 ±7*1.00, —4.72, —2.35 ±7*3.96. The dominant poles are 
therefore correct, at least to within 0.5%, but the remaining poles are not 
far removed from the dominant poles. The polynomial having zeros that are 
these closed-loop poles is 

p(s) = s 6 + 12.4s 5 + 75.5s 4 + 267.5s 3 + 489.5s 2 + 484.4s + 198.5. 

The feedback vector k is given by 1 — k'(sl — F)~ x g = p(s)ly/(s), and 
is readily found to be [-198.5 -484.4 -489.5 -268.5 -76.5 -11.4]. 

Two warnings are perhaps appropriate at this stage. It will probably 
not have escaped the reader’s notice that the entries of the vector k in the 
two examples are large, relative to, say, the entries of F and g. This means 
that the magnitude of the control arising from feeding back k'x may be large, 
even though x and Fx are small. Equivalently, there will be a tendency for 
saturation at the plant input to occur as a result of the entries of k being 
large, and, of course, the larger these entries are, the more likely is the occur- 
rence of saturation. It is not surprising that this phenomenon should occur, 
in view of the fact that if p is large in the performance index (5.4-9), nonzero 
states are weighted more heavily than nonzero controls. Therefore, larger and 
larger p will tend to make the entries of k larger and larger, and make satura- 
tion more and more possible (Problem 5.4-5 asks the student to connect 
variation in p with variation in the entries of k .) 

Reference to (5.4-13), the equation that yields the minimum p guarantee- 
ing a dominancy factor of 10, shows that the more nondominant poles there 
are, the larger p must be. Therefore, the possibility of saturation is lowered 
by having, say, only one nondominant closed-loop pole. 

The second warning concerns the interpretation of dominancy. Suppose 
a transfer function is given with n poles, n' of which are dominant. Suppose 
also that there are n" dominant zeros, and suppose we are interested in the 
step response of a system with this transfer function, Then, unless ri* < n\ 
or, better, n" < n\ the residues associated with the nondominant poles will 
not be small relative to the residues associated with the dominant poles. 
Accordingly, the step response will only be free of a sizable component of 
fast transients if the upper bound on n" is met. Hence, the term nondominant 
pole really has the fuller meaning: the associated component of transient 
response is not only very fast, and therefore dies away quickly, but is also 
small, and therefore negligible. We conclude that any attempt at pole posi- 
tioning to obtain satisfactory transient response should take into account the 
relative number of dominant zeros and dominant poles. [The effect of state- 
variable feedback on the zeros of a transfer function will be discussed shortly.] 
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We shall now discuss a straightforward extension of the closed-loop pole 
specification idea to multiple-input systems. To apply the preceding design 
procedure, we adopt an ad hoc procedure for converting the multiple-input 
system into a single-input system. 

The procedure relies on an observation of [5] that if [F, (7] is completely 
controllable, where F is n X n and G is n X m, and if F has distinct eigen- 
values, then [F, Gy\ is “almost surely” completely controllable, where y is 
an arbitrary nonzero m vector. If F does not have distinct eigenvalues, the 
observation may still be true. There is, of course, a technical meaning for 
“almost surely,” which we shall not bother about here. We shall take the term 
to mean what we would normally understand by it. That is, with [F, G] com- 
pletely controllable, if we choose an arbitrary y such that by some rare 
chance [F, Gy] is not completely controllable, then an infinitesimal change in 
y will ensure that [F, Gy] is completely controllable. 

Returning to the closed-loop pole specification problem, let us suppose 
we are given a completely controllable m input system that 

x = Fx + Gu y (5.4-14) 

where we label the input u to distinguish it from the input u of a single-input 
system, which we define in a moment. Let y be an arbitrary m vector, such 
that [F, Gy] is completely controllable. The equation 

x = Fx + Gyu (5.4-15) 

defines a completely controllable single-input system. For the moment, 
we focus attention on (5.4-15) rather than (5.4-14). We can find a large 
constant p, and an n vector d (which then defines a matrix Q = dd '), such that 
the optimal control law u = k'x minimizing the performance index (5.4-9) 
gives an optimal closed-loop system 

x = (F+Gyk')x (5.4-16) 

with closed-loop poles approximating the closed-loop poles desired for the 
m-in put system. 

Now we observe that if the control law u = yk'x is used for the open- 
loop scheme (5.4-14), the resulting closed-loop scheme is also (5.4-16), and, 
consequently, the desired closed-loop poles are achieved. The control law 
u = yk'x is not strictly an optimal control for (5.4-14), but in the light of the 
closed-loop scheme (5.4-16) being optimal [admittedly, for a different open- 
loop system than (5.4-14), but this is somewhat irrelevant], it would be expected 
that some of the ancillary benefits of optimality, including those yet to be 
discussed, would accrue. 

Problem 5.4-7 asks for a design scheme for generating desired closed- 
loop poles, which also results in a truly optimal multiple-input closed-loop 
system. 

A second , but sometimes unsatisfactory, approach to achieving desired 
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closed-loop pole-positions is the following. We suppose we are given a com- 
pletely controllable system with no restriction on the number of inputs: 

x - Fx + Gu. (5.4-17) 

Furthermore, given a set of desired closed-loop poles, the number of poles 
being equal to the dimension of F, we construct a matrix F the set of eigen- 
values of which coincides with the set of the these poles. We then form the 
following performance index: 

poo ^ 

V(x(t 0 ), u(-), * 0 ) = [u'Ru + p(x — Fx)'Q(x — Fx)] dt. (5.4-18) 

Here, F, Q , and p satisfy the usual constraints. The presence of the second 
term in the integrand of the performance index (5.4-18) serves to force the 
optimum state trajectory to be close to the solutions of x — Fx, which 
amounts to forcing the closed-loop poles to be near to the eigenvalues of F. 

To minimize (5.4-18) is quite straightforward. First, the quantity x in 
the integrand is replaced by Fx + Gw, using (5.4-17), which allows the inte- 
grand to be written in the form 

u'R t u + 2x'S y u + x'Q x x, 

with Q x — S l R x 1 S\ nonnegative definite and R x positive definite. With this 
rewriting of the integrand, determination of the control law is straightforward, 
using the ideas of Chapter 3, Sec. 3.4. 

The minimizing of (5.4-18) will lead to closed-loop poles that approxi- 
mate, but in general do not equal, the desired values. As p becomes larger, 
the degree of approximation will clearly improve. 

The potential disadvantage of this form of optimal control lies in the 
presence of the cross product term x'S t u in the integrand of the performance 
index. As noted in the last problem of the immediately preceding section, the 
presence of such a term may destroy the pleasing properties associated with 
phase margin and gain margin. It also destroys nearly ail the other pleasing 
engineering properties to be discussed in the next two chapters. 

In the remainder of this section, we shall consider what happens to the 
zeros of a system where optimal linear feedback of the states is introduced . 
We consider the situation of Fig. 5.1-2, where the prescribed plant has transfer 
function h’(sl — F)" 1 #, and a feedback k'x is applied. The transfer function 
between input and output after applying feedback is h'(sl ~ F — gk’Y'g. 
Therefore, we are concerned with the answer to the following question: 
How do the zeros of the transfer function h’(sl — F — gk')' l g differ from 
those of the transfer function h'(sl — Ff'gl The answer is that the zeros of 
the two transfer functions are the same, as we shall now show. 

Suppose that det (si — F) = y/(s) and that det (si — F - gk') = p(s), 
and let polynomials l x (s) and l 2 (s) be defined by 



h'(sl ~ F)~ l g 



IM 

y/(s) 



h '(si - f - gk'Y'g = 

p(s) 



(5.4-19) 
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We have to show that l ,(s) = l 2 (s). Now 

h'(sl -F- gk'Y'g = h'(sl — F — gk’rHsI - F)(sl - F)~ l g 

= h'(sl -F-gk'Y 1 m -F- gk') + gk'}(sl -F)-'g 
= h'(sl — F)~ l g + h\sl -F- gk')-'gk'(sl - F)~'g. 

Therefore, 



h'jsl - F — gk'Y'g — ~ F ^‘ g 

s ’ s 1 — k(sl — FY x e 



(5.4-20) 



Now the definition we have first adopted for/?(,s) coincides with that of (5.4-6), 
and with p(s) so defined, Eq. (5.4-4), repeated here for convenience, is also 
valid: 



l-k'(sI~FY' g = ^- (5.4-21) 

When Eqs. (5.4-19) and (5.4-21) are inserted into (5.4-20), the result l { (s) = 
l 2 (s) is immediate. 

In reality, the optimality of k is not a relevant factor in the preceding 
argument. Thus, we have established the important result that state feedback 
(optimal or nonoptimal) leaves invariant the zeros of the open-loop transfer 
function. 

As a note of warning, we draw attention to the need not to confuse the 
transfer function h'(sl — F)~^ g of Fig. 5.1-2 with the transfer function of the 
open loop part of the overall scheme, viz., — k'(sl — F)~'g , some of the prop- 
erties of which we studied at some length in the last section. Although the 
zeros of this transfer function and its closed-loop counterpart ~k\sl — F 
— gk)~ l g are the same, they are, of course, both dependent on k. 

As references for the material of this section, we may list [1] and [6], 
which discuss the properties of the closed-loop system poles when p approaches 
infinity in the performance index (5.4-9); [7], which discusses the second 
pole-positioning procedure presented; and [8], which demonstrates the 
invariance of the zeros of a transfer function with state-variable feedback. 
Chapter 7, Sec. 7.4, considers further the problem of pole positioning in 
optimal systems. 



Problem 5.4-1. Consider a system 




with performance index J f0 ( u 2 + x\ -j~ x\)dt . Using the polynomial factorization 
technique, find the optimal control. 



Problem 5.4-2. With the same system as in Problem 5.4-1, suppose the per- 
formance index is now J T 0 (» 2 + 900x1 + 900x1) dt. What are approximate optimal 
closed-loop poles? Compare the approximate pole positions with the actual ones. 
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Problem 5.4-3. Consider a system with transfer function w(V) = \/s 3 (s + l) 3 . 
Write down a 6 x 6 matrix F, and vectors g and h such that w{s) — h'(sl — F)“ l g. 
Then define and solve an optimization problem so that with state variable feedback, 
the new system transfer function becomes approximately a/(s + 1 )(s 2 +5 + 1) for 
some constant a. [Hint: A choice for F of the companion matrix form and for g' 
of the form [0 • • * 0 1] may be straightforward.] 

Problem 5.4-4. Consider a system x ~ Fx -f gu with performance index 
\Z( u2 + px'Qx)dt for nonnegative definite Q and positive p. Suppose that with 
D any matrix such that DD' = Q, the pair [F, D\ is completely observable. Discuss 
what happens to the poles of the closed-loop system as p approaches zero. Let 
u = k'x be an optimal control law resulting for very small p. Discuss the difference 
between the magnitude and phase of the transfer functions h'ijcol — F)~ ] g and 
h'ijcol — F — gk')~ [ g as CO varies. 

Problem 5.4-5. Suppose u = k'x is the optimal control for x = Fx + gu , 
with performance index |,’ 0 (u 2 -f px'Qx)dt. Show that as p approaches infinity, 
some, if not all, the entries of k approach infinity as fast as \/~p ". 

Problem 5.4-6. Consider the multiple-output, multiple-input system x = 
Fx + Gu, y = H'x . Suppose state-variable feedback is applied, so that the new 
closed-loop system equations are x — (F + GK')x Gu, y — H'x. Can you indi- 
cate quantities analogous to the zeros of a scalar transfer function which remain 
invariant with the specified feedback? 

Problem 5.4-7. Consider the multiple-input system x = Fx -f Gu, with per- 
formance index J, x 0 (u'u + px'Qx)dt , with the usual constraints on Q. If a = K'x 
is the optimal control law, it follows that [/ — G'(—sl — F'Y 1 K][I — K'(sl — F)~ 1 C] 
= / + pG'(— si — F')~ l Q(sI — FY ] G. Now, because of the easily established 
relation 

[/ - K'(sl - F)~ l G]~ l - / + K'(sl F- GK'Y'G , 

it is evident that the poles of the closed-loop system are the zeros of det [/ — 
K'(sl — F)" 1 G]. Assuming that, except for isolated values of s, G'(-sI ~ F')~ 1 Q(sl 
— F)~ 1 G is nonsingular, discuss a technique for choosing Q and p so that the closed- 
loop system has certain desired poles. (This is a difficult problem. See [9] for a lengthy 
discussion.) 

Problem 5.4-8. In this problem, a relation is developed between the band- 
width of a single-input, single-output, closed-loop, optimal system and some of 
the parameters appearing in the optimization problem. This allows design for fixed 
bandwidth of an optimal system, [10]. Consider the transfer function w(s) = 
h'(sf — F)~ l g, with [F, g] completely controllable and [F, h] completely observable. 
Suppose a state feedback law is found that minimizes the performance index, 

Foo 

[u 2 + p(h'x) 2 ] dt. 

J tv 

Let w’c L (s) denote the corresponding closed-loop transfer function, and let co 0 be 
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the frequency for which 

|Wc L (iO>o)| 2 =i|WcL(0)P. 

Show that co 0 may be determined as the frequency for which 

I HjCOo) | 2 = p + (2/1 w(0) j 2 )' 

[Note the simple form of this result for the case when w(s) has a pole at s = 0.] 
Explain what happens as p approaches infinity, considering first the case when 
w(s) is minimum phase — i.e., its poles and zeros all have negative real parts. 

Problem 5.4-9. Suppose you are given a single-input, single-output system, 

x — Fx 4 - gu y = h'x. 

Suppose also that it is desired to have the output behave in accordance with the 
law y = — ay for a certain positive tt. Consider the effect of using the performance 
index 

V = [ u 2 4 - p(y 4 - tty) 2 ] dt. 

J to 

Explain what happens as p -> oo, considering the following two cases. 

1. The zeros of h'(sl — F)' ! g are in Refs] < 0. 

2. The zeros of h\sl — F)~ l g are arbitrary. 

Problem 5.4-10. Discuss the case when the polynomial m(s) in Eq. (5.4-12) 
is such that [F, d] is not completely observable. [Hint: Use the fact that the dynamics 
of the unobservable states are defined by common zeros of m(s) and y'COd 
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6 



INSERTION OF NONUNEARITIES 
IN OPTIMAL REGULATORS 



6.1 TOLERANCE OF NONLINEARITIES IN THE FEEDBACK 
LOOP 



Many nonlinear physical systems may be described by linear models; 
analysis may then be carried out on the linear model, and the conclusions 
of this analysis may be used as an approximate guide to the behavior of the 
actual nonlinear system. If the nonlinearity of the physical system is small in 
some sense, then it would be expected the physical system and the model 
would behave similarly. If the nonlinearity of the physical system is not small, 
it is, of course, still desirable and sometimes possible to make predictions, 
based on analysis of an associated linear model, which will give information 
about the physical system. 

In this section, we shall consider the introduction of gross nonlinearities 
into regulator systems, and we shall be concerned with establishing stability 
properties of the resulting nonlinear systems from certain properties that 
we know about (linear) regulator systems. 

The scheme we consider is shown in Fig. 6.1-1. We suppose we are given 
a completely controllable system, 

x = Fx + Gw. (6.1-1) 

We suppose also that there is an associated performance index 

J OQ 

e 2ot '(u'w + x'Qx) dt (6.1-2) 

to 
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Fig. 6.1-1 Optimal system with inclu- 
sion of nonlinearities. 



where a is a nonnegative constant, 0 is nonnegative definite symmetric, 

and if D is any matrix such that 0 = DD' f the pair [F, D] is completely 

observable. [We could, of course, consider a more general performance index 
than (6.1-2) — one where u'u is replaced by u'Ru. But we can note that the 
associated minimization problem is equivalent to one of the form defined by 
(6.1-1) and (6.1-2), because the minimization of J“ e 2at (u'Ru + x'Qx) dt 
subject to x — Fx + Gu is equivalent to the minimization of j™e 2at (i/ 1 u 1 
~\-x'Qx)dt subject to x = Fx + GR~ 12 u 1 under the identification u i = 
R h2 u.] The matrix K a appearing in Fig. 6.1-1 defines the optimal control for 
the minimization problem of (6.1-1) and (6.1-2) — i.e., 

u = K' a x (6.1-3) 

minimizes (6.1-2). We recall that the matrix K a is given by 

K = -P.G, (6.1-4) 

where P a is the unique positive definite solution of 

P a (F + a/) + (F' + a I)P a - P a GG'P x +0=0. (6.1-5) 

Figure 6.1-1, of course, shows a variant on the optimal control arrangement. 
If the nonlinear feedback block were, in fact, a block that had the same 
outputs as inputs, Fig. 6.1-1 would represent a truly optimal system. But 
we allow a variation from the optimal arrangement; we shall permit a wide 
class of nonlinearities, to be defined quantitatively. 

If the external input to the arrangement of Fig. 6.1-1 is zero, it is clear 
that there would be no difference if the nonlinearities were moved to the right 
of the summing point — i.e., between the summing point and the block with 
transfer function matrix —K' a (sl - F) _1 <7. They could then be regarded as 
being located at the inputs of the system x = Fx + Gu rather than in the 
feedback loop. Both interpretations are equally valid. Nonlinearities in the 
input transducers of physical systems are quite common (e.g., because com- 
monly high power levels are present, and saturation may occur). 

We shall now define the class of nonlinearities permitted. Then we shall 
discuss the stability of the system (6.1-1) with nonlinear feedback. The non- 
linear block (f >( •) will transform an m vector y, where m is the dimension of 
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w, into an m vector <j>(y). In addition, we require that <j)(y ) be continuous and 
that the following constraints on $(•) hold for all y: 

(i + c)y'y < y'<l>(y), (6.1-6) 

where e is an arbitrary small positive number. This is not an unreasonable 
condition to investigate, because the gain margin properties of regulators 
discussed in a previous chapter indicated (at least for single-input systems) 
that if $(•) were linear, then it could satisfy (6.1-6) without a loss in stability 
of the closed-loop system. Note that (6.1-6) does not require the zth output 
of the nonlinearity $ t (y) to depend purely on the zth input y t ; in other words, 
the nonlinearity can somehow “mix up” its inputs. Nevertheless, the situa- 
tion where 0/y) depends solely on y ( for each z is the most common. Then 
(6.1-6) becomes 

* i = 1, 2, m (6.1-7) 

when 0. Equation (6.1-7) implies that the graph of $ t (y t ) versus y t 
for each i must be within any sector bounded by straight lines of slope £ 
and co; in Fig. 6,1-2, the shaded sector is a permissible region in which the 



Fig. 6.1-2 Permissible region for graph 
<t>i(yd versus v;. 




i 

7 



graph of <f>i(yd versus y t should lie. Notice that this figure illustrates that 
4>i( 0) = 0 for each /—in fact, the more general constraint (6.1-6) and the 
continuity requirement imply that 0(0) = 0. Now, with no external input, 
the scheme of (6.1-1) is represented by 

x = Fx + G<f>{K’ a x). (6.1-8) 

We need to examine the stability of (6.1-8) for all initial conditions. Let 
us take as a Lyapunov function 

V{x) = x’P a x (6.1-9) 

and observe that F(x) satisfies the requirements listed in Theorems A and 
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B of Chapter 3, Sec. 3.2, viz., V{x) is positive definite and approaches infinity 
as x'x approaches infinity. 

Next, we need to evaluate V. We have (with the various steps to be 
explained), 

V(x) = x'P a x + x'P a x 

= x'(P a F + F'P a )x + 2 x'P a G<l>(Kx) 

= x'[P«(F + a/) + (F' + a I)P a - P a GG'P a + Q]x 

-2a x'P x x - x'Qx + x'P a GG'P a x + 2 x'P a G<f>(-G'P a x) 

= -2a x'P a x - x'Qx + x'P a GG'P a x + 2x'P a G(j)(-G f P a x) 

< -2a x'P a x - x’Qx + x f P a GG'P a x - 2Q- + €)x'P a GG'P* x 

= —2a x'P a x — x'Qx — 2ex'P tt GG'P a x. (6.1-10) 

The second equality follows from (6.1-8), the third by simple rearrangement 
and the relation P a G = — K a , and the fourth from Eq. (6.1-5). The inequality 
follows from (6.1-6), and the final equality again by rearrangement. Conse- 
quently, 

V(x) < - x'Qx - 2a x'P a x - lex'K^x (6.1-11) 

At this stage, we shall separate our discussion of the cases a = 0 and 
a > 0. First, for a = 0, we have from (6.1-11) that 

V(x) < — x'Qx — Iex'KqK'qX. (6.1-12) 

Certainly, therefore, V is nonpositive, and thus (6.1-8) is at least stable. But 
we shall also argue that V(x) is not identically zero along a nonzero trajec- 
tory of (6.1-8), and thus asymptotic stability follows by Theorem B, Chapter 
3, sec. 3.2. The argument is by contradiction. If V{x) = 0 from some time 
h on, then K' 0 x == 0 from t * on, and thus $(K’ 0 x) = 0. Then (6.1-8) becomes 
the same as x = Fx. But V(x) = 0 also implies x'Qx = 0, and thus x\t x ) 
exp [F\t — t x )]Q exp [F(t — t x )]x(t x ) = 0 for all t > t x . This contradicts 
the complete observability of the pair [F, D], where D satisfies DD' = Q. 

Therefore, V(x) = 0 for t > t x is impossible, and asymptotic stability truly 
prevails. 

Now consider the case a > 0. From (6.1-11) we have immediately 

< —2<xx'P a x = ~2aV(x), (6.1-13) 

which implies that V(x(t)) < e~^V(x(0)) by the following simple argument. 
Let a{t) be a nonnegative function such that V(x) = — 2aF(*) — a(t). The 
solution of this differential equation is well known to be V(x(t)) = e~ 2at V(x(0)) 

- J r 0 e- 2 *‘e 2 «‘a( t) Jt, from which the desired conclusion is immediate. 
Rewriting V(x(t)) < e~ 2at V(x(0)) as 

x'(t)P a x(t) < e- 2 *‘x'(0 )P a x(0), 



(6.1-14) 
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we see that since *(•) is the solution of (6.1-8), then, using an obvious defini- 
tion of degree of stability, (6.1-8) possesses degree of stability a. 

We recall that with a — 0, the optimal system is known to be asymp- 
totically stable, and with a > 0, the optimal system is known to have degree 
of stability a. Thus, the introduction of the nonlinearity $(•) does not affect 
these stability properties of the closed-loop system. 

For single-input systems, there is a connection with the material of 
Chapter 5, Sec. 5.3, where we discussed the question of the gain margin of 
the optimal regulator. We recall from that section that if a constant (linear) 
gain ^ is placed in the feedback loop of an optimal regulator, the stabil- 
ity properties of the regulator are not disturbed. Now a constant linear gain 
greater than £ is a special case of a nonlinearity confined to the sector of Fig. 
6.1-2. Therefore, the earlier result of Chapter 5 is included within the more 
general framework of this section. 

We shall now show how the constraint of (6.1-6) may sometimes be 
relaxed even further, in the sense that stability results may follow for the 
following revised constraint: 

P/y <y'<f>(y) (6.1-15) 

for certain /? < Since the constraint \ for the case when a is nonzero 
leads to a closed-loop system with degree of stability a, we would expect that 
for a certain range of /? < £, asymptotic stability would still prevail, but 
perhaps with a smaller degree of stability. Indeed, we shall verify this prop- 
erty. 

For the moment, we do not restrict a. As before, we consider the closed- 
loop system (6.1-8), with Lyapunov function V(x) — x'P a x. The second 
equality in (6.1-10) follows immediately. We repeat it here for convenience: 

V(x) = x\P a F + F'PJx + 2 x'PM(K'«x). (6.1-16) 

Using P a G = —K a9 and the new constraint (6.1-15), we have 

V(x) < x'(P a F + F'P a )x - 2 fix 'K a Kx 

from which we can derive, using (6.1-5), 

V(x) < - x'Qx - lax'P^x + (1 - 2$)x' K a K' a x. (6.1-17) 

Now we shall consider separately the cases a = 0, a > 0. When a = 0. 
(6.1-17) is 

V(x) < -x'[Q - (1 - 2fi)K 0 K’ 0 ]x. (6.1-18) 

Now, if < i is chosen so that Q - (1 - 2fi)K 0 K' 0 is nonnegative definite, 
stability of (6.1-8) will follow, and if G — (1 — 2fi)K Q K' 0 is positive definite, 
asymptotic stability will follow. Do such /? exist? Possibly not, but if Q is 
positive definite, certainly there are values of ft less than £ for which Q — 
(1 — 2fi)K 0 K' 0 is positive definite. For a symmetric matrix A , let A max (^) 
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denote the maximum eigenvalue and A min (/4)the minimum eigenvalue; clearly, 
A max (A)I > A > X min (A)f. Then a sufficient condition on /} ensuring that 
Q — (l — 2 fi)K 0 K' 0 is positive definite is 

A min (0 - (1 - 2) 3)X max (K 0 K' 0 ) > 0. (6.1-19) 

But if, on the other hand, Q is singular, then there may be no fl less 
than \ for which Q — (1 — 2$)KJP % is even nonnegative definite, although 
with f} = nonnegative definiteness follows. 

Now consider the case a > 0. Equation (6.1-17) becomes 

V(x) < -x'[Q + 2a P a - (1 - 20)K a K' a ]x (6.1-20) 

and certainly now (because P a is positive definite) there exist /? less than ^ 
such that Q + 2a P a — (1 — 2fi)K a K' a is positive definite. A sufficient condi- 
tion on p that ensures this is that 

A min (g + 2a P a ) - (1 - 2 p)X m&x (K A K’ a ) > 0. (6.1-21) 

If for some fixed ft, Q + 2a/ > a — (1 — 2 P)K OL K' OL is positive definite, it is pos- 
sible to compute a lower bound on the resultant degree of stability of the 
closed-loop system. Let y be such that 

KdQ + 2aP a - (1 - 2 p)K a K' a ] = yX m&x (PJ. (6.1-22) 

Then (6.1-20) implies V(x) < — ?A max (P a )x'x < ~ yx'P^x — i.e., V(x) < 
— yV(x). So yjl is a lower bound on the degree of stability of the closed-loop 
system. 

Smaller /? lead to smaller y\ thus, as /? becomes smaller, there is a con- 
tinuous exchange between the amount of tolerable nonlinearity and the degree 
of stability. This exchange eventually terminates when the amount of tolerable 
nonlinearity exceeds that which will permit stability to be retained. 

Implicitly in the foregoing discussion, we have assumed that the non- 
linearity (j> is time invariant — i.e., for all t x and r 2 , 0(^(L)) = <t>{y(t z )) if 
y(t i) — But a speedy review of the calculations will soon show that 
time invariance of $(•) is not essential; it may just as well be time varying. 
Thus, in the special case of a single-input system, so long as a time-varying 
nonlinear gain t) has the property that \+e < </>()>, t)/y for all nonzero 
y, stability of the closed-loop system follows. 

It is possible to view this, and indeed the corresponding multiple-input 
result, as following from an application of the circle criterion [1]. To state 
this criterion, consider the feedback arrangement of Fig. 6.1 -3(a) and assume 
the external input is zero. Suppose also that the time-varying nonlinear gain 
element with input y has output k(( , y)y, and that the graph of zc(/, y)y versus 
y for all t is confined to a sector itself inside a sector defined by straight lines 
of slope k, and *r 2 with 0 < k x < /c 2 , as shown in Fig. 6.1-3(b). The circle 
criterion states that if the closed-loop system is asymptotically stable for all 
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Fig. 6.1-3 Diagrams illustrating application of circle criterion. 



constant linear gains k with k, <rc < k z , and if the Nyquist plot of w(jco) 
does not enter the circle on a diameter defined by the points — (1 /*,) -f y'O, 
— (ll K i) +y’0, as shown in Fig. 6.1-3(c), then the closed-loop system with 
k({, >’) as described is asymptotically stable. 

In the special case of an optimal regulator, we recall from Chapter 5, 
Sec. 5.3, that the Nyquist plot of -k'«(jcoI - F)~'g does not enter the circle 
on diameter 2 +./0, 0, and at the same time the closed-loop system of Fig. 
6.1-4 with zero external input is asymptotically stable for all constant linear 



Fig. 6.1-4 Closed-loop system with 
feedback gain k . 




gams k > l Therefore, the requirements of the circle criterion are satisfied 

with at, k 2 = °°, and the conclusion is that nonlinear time-varying gains 

can be tolerated of the type described earlier in our Lyapunov function analy- 
sis of stability. 

There is another important result concerning the exchange of degree of 
stability and tolerance of nonlinearities $(•) or time-varying gain elements 
K(t). If <£(•) is considered as a time- varying gain, then the degree of stability 
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can be exchanged with rate of change of gain variations. From [2] and [3], 
we shall quote general results that may then be applied directly to the optimal 
control case. 

Using the notation introduced for a description of the circle criterion, 
we assume that the closed-loop system w(s)/( 1 + tcw(V)) has a degree of 
stability a for all constant linear gains k with 0 < k { < k < k 2 for some con- 
stants k j and k 2 . We introduce the quantity k defined for fixed but arbitrary 
T by 



f 1 

k = SU P AT 

r>D 




*(t)(k 2 - *,) 
[k(t) - K 2 ][k(t) -- ATj] 



dx. 



This quantity k is in some way a measure of the time variation of k((). The 
key result is that for k < a < a, the degree of stability of the closed-loop 
system is (a — a). This result provides a direct and continuous trade-off 
between degree of stability and tolerance of time variation. It has immediate 
application to the optimal regulator with a prescribed degree of stability 
a, where by taking k x < \ and k 2 = we can perhaps extend the regions of 
known degree of stability. 

So far in this section, the presence of nonlinearities has been considered 
at one point only of an optimal control system (provided we regard input 
nonlinearities as being the same as feedback nonlinearities). The natural 
question arises as to whether any remarks can be made concerning the effect 
of having nonlinearities at other points of a nominally optimal system. 
Certainly, a '‘small” amount of nonlinearity can be tolerated anywhere, 
without affecting asymptotic stability, when the truly optimal system is itself 
asymptotically stable. This is a basic result of stability theory (e.g., see [4]). 
Reference [5] actually computes sector bounds on arbitrary nonlinearities, 
which, if not exceeded, will retain asymptotic stability. 

The proof of the result that gross input nonlinearities are tolerable for 
optimal regulators will be found in [6] for the scalar input case. The vector 
input case is covered in [5], which indicates the connection with the circle 
criterion for multiple-input systems. Results are also possible for time-varying 
systems, for which [7] should be consulted. 



Problem 6.1-1. Consider a single-input, completely controllable system 
x ~ Fx -f~ gu y and let u = k x be a control law that minimizes a quadratic perfor- 
mance index and stabilizes the system. Suppose, however, that this control law is 
not used; rather, the law u = p Q sat [Mk'x)lp 0 ] is used where 1 ^ p 0 < oo, 
1 < Mo < oo, and sat y = yfor \y\< 1, = +1 fcry > 1, and = -1 fory < -1. 
Show that for sufficiently small *(0), depending on p Q and /i 0 , the closed-loop sys- 
tem is asymptotically stable. (See [6] for a full discussion of this result.) 

Problem 6.1-2. Consider the feedback arrangement of Fig. 6.1-1 with K a , 
Fy and G as earlier described, and <f )( •) constrained as in Eq. (6.1-6). Suppose that 
an external input */(•) is applied, with u'(t)u(t) < M for all t and some constant 
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M. By taking V = x'P^x and computing a modified V to take into account the fact 
that «(•) is no longer zero, show that the states remain bounded, and that this bound 
depends only on M and x(0) and not on the particular «(•) adopted. Discuss the 
application of this result to the consideration of stability of systems with nonlinear 

feedback with hysteresis, when the external input is identically zero. (This is a hard 
problem.) 



6.2 RELAY CONTROL SYSTEMS 



In this section, we consider some general properties of control systems 
containing relays. The results will be applied in the next section to consider 
the effect of placing relays in the control loop of an otherwise optimal system. 
In a later chapter, the theory will be applied to certain optimal systems where 
the optimal control law may be realized using linear gain elements together 
with a relay. 

Our discussion here will start with the definition of a relay system that 
will take into account the notion of chattering. Following this, stability 
results will be presented. Our initial discussions make no restriction of time 
invariance or linearity of the nonrelay part of the system, but do restrict 
consideration to single-input, single-output systems. (Results for multiple- 
input, multiple-output systems are complex, and not well detailed in the 
literature.) The stability discussions will be for restricted classes of systems 
and will serve as background for considering the effect of introducing relays 
into nominally optimal linear regulators. 

To define relay systems and the notion of chattering, we start by consider- 
ing the scheme of Fig. 6.2-1, with the equation 

* =/[*, sgn (h'x), t] (6.2-1) 

where 

sgn o{t) = +1 for a(t) > 0, or a(t) = 0 with a{t - e) > 0 

= — 1 for a(t) < 0, or o(t) = 0 with a{t — e) < 0 

= 0 otherwise (6.2-2) 

and e is an arbitrarily small positive number. 




fig. 6.2-1 Basic relay feedback system. 
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At this point, several comments are in order. First, we could have consid- 
ered a more general relay scheme than that of (6.2-1), where fix is replaced 
by //(x, t ) — a scalar, but not necessarily linear, function of x. The analysis 
would be almost the same. For our purposes, though, the slightly restricted 
form of (6.2-1) is more convenient. Second, we shall subsequently take 
specializations of (6.2-1), requiring it to be a linear time-invariant system, 
save for the relay, and to be a linear time-invariant system, save for input 
nonlinearity and the relay. There is little point in making the specialization 
at this stage, however, which we postpone until just before a discussion of 
stability. Third, the definition of (6.2-1) and (6.2-2), which is due to Andre 
and Seibert [8], may seem unreasonably complex. In particular, it may be 
felt that it would be sufficient to always define sgn 0 as 0, rather than some- 
times — 1, +1, or 0, as in (6.2-2). However, this is not so, and indeed (6.2-2) 
as it stands even requires special interpretation to avoid certain difficulties, 
which we shall describe shortly. 

Referring to (6.2-1), we see that the switching surface or hyperplane 
h'x — 0 (which is a line where x is a 2 vector, a plane when x is a 3 vector, 
etc.) divides the state space into two regions, in one of which the control to 
the system x = /(x, w, /) is +1, in the other — 1. The scalar quantity o = h'x 
measures the distance of a point x from the nearest point on the switching 
surface, with due attention paid to algebraic sign, whereas the scalar a 
measures the rate at which an arbitrary point x on a trajectory is approaching 
the switching surface. 

Consider an arbitrary point w on the switching surface, together with the 
values of a and 6 near w. Assuming for the moment 6 is nonzero, several 
situations arise in respect to the possible signs of a and a. In the a > 0 
region, & may be positive or negative, and likewise in the a < 0 region. 
Thus, any of the possibilities shown in Figs. 6.2-2(a) through 6.2- 2(d) may 
hold. These figures show typical trajectories on each side of the switching 
surface; note that these trajectories have directions consistent with the signs 
of &. 

In cases (a) and (d) of Fig. 6.2-2, switching is instantaneous. Trajectories 
approach the switching surface from one side, cross it, and move away from 
it on the other side. In case (, h ), no trajectories approach the switching surface, 
and thus no switching occurs. However, if a system is started in a state w 
on the switching surface, the subsequent trajectory is not uniquely defined, 
as two trajectories leave vv, extending into the a > 0 and a < 0 regions. In 
case (c), trajectories in both the o > 0 and a < 0 regions will head toward 
the switching surface. No trajectories can leave the point u>, and thus the differ- 
ential equation (6.2-1) apparently cannot be solved beyond this point. For 
this reason, points such as w are termed “endpoints.” 

This raises an obvious difficulty: If in Fig. 6.2-2(c), a trajectory reaches 
the point w, mathematically the system behavior cannot be defined beyond 
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(c) (d) 

Fig. 6.2-2 Possible trajectories at switching time. 



the time when w is reached. Physically, of course, something must happen, 
and thus we are faced with a paradox. This has been resolved in [8], where 
one or both of the following variations in relay operation are assumed to take 
place. 

1. Operation of the relay is delayed by a very short time r. Then if a 
changes instantaneously from a positive to a negative value at time 
t — 0, the relay output changes from +1 to —1 at time t ~ t. 

2. There is hysteresis in the relay. It operates when a has changed sign 
and passed to a slightly nonzero value, e > 0 for a sign change from 
negative to positive, and — e < 0 for a sign change from positive to 
negative. 

The effects of (1) and (2) are similar. Supposing that mechanism (l) is 
present, we see that a trajectory heading for w will continue on past w to 
the opposite side of the switching surface. At time t, after crossing the switch- 
ing surface, the sign of the control will change, causing the trajectory to head 
back toward the switching surface. However, on reaching the switching sur- 
face, the trajectory will again carry on across it as the sign of the control 
again will not change for a time t. The net result is that the trajectory zigzags 
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across the switching surface at a rate determined by r, and the smaller x is, 
the higher will be the frequency of crossings of the switching surface. 

Under these conditions, the solution of (6.2-1) is said to exhibit a chatter- 
ing mode of behavior , and this mathematically predicted mode of behavior 
turns out to be in accord with physical observations. Moreover, the theory 
covers the situation where a can be zero at or near endpoints such as w , a 
situation hitherto excluded. 

Let us now give a quantitative analysis. Instead of (6.2-1), we take as 
the describing equation 

x = f[x{t\ sgn h f x(t — r), *]. (6.2-3) 

Adopt the notation f + to denote f(x, 1, t) and /- to denote /(x, —1, /). We 
note first a condition for a state x — w on the switching surface h'x = 0 at 
time t to be an endpoint. Referring to Fig. 6.2-2(c), we see that a > 0 implies 
d < 0 in the vicinity of w if h'f + < 0; likewise, a < 0 implies d > 0 in the 
vicinity of w if h'f~ > 0. Note that these conditions are only sufficient for w 
to be an endpoint. If hf + = 0, for example, w may or may not be an endpoint. 
Restricting ourselves to the simpler case, though, we note the endpoint condi- 
tion : 

h'f + < 0 and hf ' > 0. (6.2-4) 

We shall now consider the chattering mode solution of (6.2-3), and we 
shall study in particular what happens as r approaches zero. 

Suppose that for t < t l9 the state lies in the region h'x < 0, and that at 
t x , we have x(^) = w— i.e., the trajectory meets the switching surface. Define 
x 2 = x(t x + r), which is the value of the state vector when switching of the 
control takes place from — 1 to +1, and let x 3 = x(t x + At) be the state 
vector when the trajectory next intersects the switching surface, where we are 
implicitly assuming that w is an endpoint. 

In the interval [t x , t x + r], we have sgn h'x{t ~ t) = —1, so that 



x 2 = w + xf~ + 0(t 2 ). 

For t > t , + t, the control is u = +1. Hence, 

*3 = *2 + (At - x)f* + 0[(Af - t) 2 ] 

= W + A tf + +T(f-~ f + ) + 0(t 2 ) + 0[(A/ - T) 2 ]. 

Multiplying this equation on the left by h’ and recalling that and w are 
both on the switching surface h’x = 0, we obtain 



Furthermore, 



A t 



Af -- T ^(/--/ + ) . 

h’f* 



vrr— 



h\r - n 
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Clearly, as t -> 0, the system trajectory approaches arbitarily close to the 
trajectory defined by 

* =/ + -(/' -f + ) hXf h ' f l f ~y (6.2-5) 

As one might expect, as t approaches zero, the system trajectories are 
restricted to leaving the switching surface by a smaller and smaller amount, 
and in the limit must remain on it. This is borne out by evaluating h'x using 
(6.2-5), which is immediately checked to be zero. 

Similar arguments carry through when hysteresis is postulated. Let us 
summarize the salient points. 

1. The relay system (6.2-1) is inadequately described by the definition 
(6.2-2), because of the possible phenomenon of endpoints. 

2. “Endpoints” will occur if the constraint (6.2-4) holds. 

3. When the system reaches an endpoint, its physical behavior is one of 
chattering. The states move according to (6.2-5), and remain on the 
switching surface h'x = 0 when infinitely small time delays in switch- 
ing or hysteresis in the relay are postulated. If and when (6.2-4) 
subsequently fails, the system leaves the chattering mode, reverting 
to its regular mode of operation described by (6.2-1) and (6.2-2). 

Linear systems with relay feedback. We shall now consider a specializa- 
tion of the foregoing results, replacing the system (6.2-1) by 

x = Fx — g sgn (h'x) (6.2-6) 

where F, g y and h are all time invariant. Figure 6.2-3 shows the arrangement, 
with the external input in the figure assumed to be zero. 



* 9 - 


iF 


— - 


h'(sl - Ff'g 




_ i 











Fig. 6.2-3 System defined by Eq. (6.2-6); the external input is 
assumed zero. 



The functions /+ and / become Fx — g and Fx + g. Accordingly, the 
condition that a point w lying on h'x = 0 be an endpoint is, from (6.2-4), 

-h'g < h’Fw < h'g, (6.2-7) 

which, in turn, indicates that endpoints occur if and only if 

h'g > 0. 



( 6 . 2 - 8 ) 
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Motion in the chattering mode is described by (6.2-5) with the appropriate 
substitutions for / + and f~\ the resulting equation is 



x ~ ^ 






(6.2-9) 



Summing up, Eq. (6.2-9) and the constraint tix = 0 completely describe 
motion in the chattering mode, with (6,2-7) defining those states on h'x = 0 
which are endpoints. 

A plausible but nonrigorous argument suggesting (6.2-9) is as follows. 
With h'x = 0, it follows that h'x = 0 or h'Fx — h'g sgn (h'x) = 0. That is, 
sgn (h'x) may be replaced by h'Fxjh'g. The equation x = Fx — g sgn (A'jt) 
then becomes precisely (6.2-9). 

The chattering associated with endpoints could be physically destructive 
of a relay, and therefore it might be thought that endpoints should be avoided 
if possible (e.g., by changing the sign of a feedback gain). But it turns out that 
if h'g < 0, implying that there are no endpoints, the resulting closed-loop 
system is always unstable. (Moreover, there exist states w on the switching 
surface such that two trajectories emanate from w, and thus the behavior of 
the system is not completely specifiable.) The remaining possibility, viz., 
h'g = 0, can arise physically. Here again, though, there must be either chat- 
tering or instability. To avoid complication, we shall not pursue this condi- 
tion here. 

To avoid chattering, a possible approach is to arrange a dual-mode 
control system: By monitoring h'x and observing when chattering starts, 
the control u ~ —sgn [h'x] can be replaced by u — — ( h'Flh'g)x , which is a 
linear control law. The system then behaves according to (6.2-9). The diffi- 
culty that arises with this arrangement is that if, owing to inaccuracies or 
disturbances, h'x = 0 fails and becomes, say, h'x — 8 for some small 5 , 
(6.2-9) does not have the property that h'x will subsequently tend to zero. In 
other words, asymptotic stability is impossible with this arrangement. 

However, there is an attractive modification. Equation (6.2-9) in the 
first instance represents behavior on the switching surface h'x = 0. Therefore, 
an alternative equation that constitutes an equivalent representation on the 
switching surface is 

i = < 6 - 2 - 10 ) 

(where a is an arbitrary constant), since the additional term is identically 
zero on the switching surface. Now suppose that a dual-mode control is 
used, based on (6.2-10) rather than (6.2-9). This would require a feedback of 
« = ~W(F + ctl)jh'g]x. As remarked, motion on the switching surface is 
unaltered, but motion in the a = h'x coordinate direction is affected. Multi- 
plying (6.2-10) on the left by h\ it follows that 



~ (h'x) — ~oc (h'x). 



( 6 . 2 - 11 ) 
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This equation implies that oc can be selected to ensure any degree of stability 
of the cr coordinate if a dual-mode scheme is implemented. This argument 
is independent of the stability of the chattering mode motion on the switch- 
ing surface h'x = 0. 

Introduction of input nonlinearity. As a second specialization of the 
general results, we replace the system (6.2-1) by 

x = Fx — g/?[sgn (h'x), t], (6.2-12) 

where /? is a (possibly time-varying) nonlinearity, with j?(l, f) and /?(— 1, t) 
bounded and continuous, and 

>0 and = £(-1, 0 < A < 0 (6.2-13) 

for all f and certain constants and /? 2 . The matrix F and vector g are 
constant. Figure 6.2-4 shows the arrangement considered, with the external 
input in the figure assumed zero. The block labeled fi is meant to symbolize 
the presence of nonlinearities, possibly time varying, in the input transducers 
of the linear system of transfer function h'(sl — F) _1 g. 




Fig. 6.2-4 Introduction of further nonlinearity into scheme of 
Fig. 2-3. 



Functions f + and f~ become Fx — gfi + and Fx + gf}~, and a point w 
lying on h'x — 0 is an endpoint if 

— h'gf}~ < h'Fw < h'gP + . (6.2-14) 

Motion in the chattering mode is again described by 

i = ( /_ © Fx - < 6 - 2 - 9 ) 

Consequently, the effect of the relay in the chattering mode is to cancel out the 
detailed effect of the nonlinearity. The actual chattering mode behavior is 
unaltered, being still governed by a linear equation, but the set of endpoints 
is affected. Motion outside the chattering mode is affected ; effectively, controls 
of p + and p~ are used in lieu of +1 and — 1. 

The use of a dual-mode controller, feeding back u = —h'Fxjh'g or 
even —[h'(F + ocI)lh'g]x when chattering is detected, is no longer acceptable. 
Such a linear feedback law will not result in the closed-loop system being 
linear because of the presence of the ft nonlinearity. 





106 INSERTION OF NONUNEARITIES IN OPTIMAL REGULATORS 



Ch. 6 



Stability. In the remainder of this section, we shall discuss the asymptotic 
stability of the closed-loop schemes (6.2-6) and (6.2-12), restricting attention 
first to (6.2-6). For the system trajectories to be well defined, we shall assume 
that the system, on reaching an endpoint, enters the chattering mode, as 
described. It is necessary to distinguish in our discussion the concepts of 
asymptotic stability (which demands boundedness and convergence of the 
trajectories to zero for a limited set of initial conditions) and global asymp- 
totic stability (which demands boundedness and convergence of the trajec- 
tories to zero for any initial condition). We shall present conditions for both 
asymptotic and global asymptotic stability, beginning with the former. 

Both frequency domain and time domain conditions for asymptotic 
stability can be presented. Although the two types of conditions are equiva- 
lent, the equivalence is certainly not obvious a priori, and this fact motivates 
the presentation of both sets of conditions. In the problems at the end of this 
section, the student is asked for illumination of the interrelation between the 
two sets of conditions. 

In [9], frequency domain conditions for asymptotic stability are stated. 
With F of dimension n x n , the transfer function h\sl — FY'g can be 
expressed in the form 



h'(sl — F)~*g = 



bn S ” 1 + * ‘ • + 
s* + a n s n ~ l + * * • + a t 



(6.2-15) 



Then necessary conditions for asymptotic stability in the vicinity of the 
origin are either (1) b n > 0 and the zeros of b n s n ' x + * * * + b x have nonposi- 
tive real parts, or (2) b n = 0, b n _ x > 0 ,a n > VA-n and the zeros of bj*' 1 
+ ■ • • + b 1 have nonpositive real parts. 

Sufficient conditions for asymptotic stability in the vicinity of the origin 
are either (1) b n > 0 and the zeros of b n s n ~ 1 + * ■ * + b x have negative real 
parts, or (2) b n ~ 0, b n ^ t > 0, a n > b n _ 2 jb n „ l , and the zeros of b n s n ~ { + • • • 
+ b x have negative real parts. 

Finally, if b„ ~ b n _ { — 0, instability is present. 

The sufficiency conditions provide a revealing interpretation using root 
locus ideas. Referring to the system shown in Fig. 6.2-5, consider those points 
on the root locus corresponding to Jetting p approach oo. It is well known 
that there are n of these points such that m approach zeros of h' (si — F)~'g 




Fig. 6.2-5 Closed-loop system with gain element p. 
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(where tn is the degree of the numerator polynomial of this transfer function), 
and n — m approach infinity. Moreover, these ( n — m) points approach 
infinity along asymptotes, which, if n — m > 3, extend into the half-plane 
Re[s] > 0. If n — m = 1 , the single asymptote extends along the negative 
real axis [corresponding to b„ > 0 in (6.2-15)]; whereas if n — m = 2 
[corresponding to b n = 0, b n _ l >0 in (6.2-15)], the two asymptotes extend 
in a direction parallel to the imaginary axis, emanating from the point 
—[a n ~ (bn-ifon-i)] + y0, which is evidently on the negative real axis. If either 
of the two sufficency conditions for asymptotic stability are fulfilled, the 
system of Fig. 6.2-5 becomes asymptotically stable for suitably large /?, and, 
of course, remains asymptotically stable for further increase in /?. The relay 
result might then have been conjectured by observing that the relay charac- 
teristic may be regarded as the limit of a characteristic of the form of Fig. 
6.2-6, as /? approaches infinity. 



Fig. 6.2-6 One form of approximation 
to a relay. 




A discussion of the time domain approach may be found in, e.g., [10] 
and [1 1]. First, consider the motion of (6.2-6) when it starts off in the chattering 
mode — i.e., when h'g > 0, and the initial state x 0 satisfies | h'Fx 0 1 < h'g 
[see Eqs. (6.2-7) and (6.2-8)]. Then it is known that the system continues in 
the chattering mode, at least until the inequality | h'Fx | < h'g fails. Now, if 
the system is asymptotically stable, it must have the property that for suitably 
small x 0 , the inequality [ h'Fx | < h'g always holds for all states x on the system 
trajectory. Consequently, for suitably small x 0 for which h'x 0 = 0, the system 
if asymptotically stable for all time moves according to 

i== ( /_ f|) Fx - (6 - 2 ' 9) 

% 

Thus, the linear equation (6.2-9) must define an asymptotically stable systemf 
for all x 0 satisfying h'x 0 = 0. It turns out (see Problems 6.2-2 and 6.2-4) 

fThe terminology is a little nonstandard. The normal definition of asymptotic stability 
demands decay to zero from any initial state near an x 0 satisfying h'x 0 = 0. We imply 
here decay to zero from precisely those initial states x 0 satisfying h'x 0 = 0. 
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that a necessary and sufficient condition for this is that the matrix 
[/ _ (g/i'// 2 'g)]F have (n — 1) eigenvalues with negative real parts. [The remaining 
eigenvalue has to be zero, as may be seen from the equation h'[I — (gtijk f g)\F 
= 0 .} 

In summary, for any initial condition on the switching surface h'x = 0 
and suitably close to the origin, the chattering mode trajectories of (6.2-6) 
are asymptotically stable, provided (6.2-9) is asymptotically stable for any 
such initial condition. A necessary and sufficient condition for this situation 
is that [/ — (gh'lh'g)]F has (n — 1) eigenvalues with negative real parts. 

What happens now for initial conditions close to the origin, but not on 
the switching surface h'x = 0? It turns out (see, e.g., [1 1] and Problem 6.2-5) 
that so long as asymptotic stability is guaranteed for initial states on the 
switching surface, it is guaranteed for all suitably small initial states, on or 
off the switching surface. Consequently, sufficient conditions for asymptotic 
stability are as follows: 

1. h'g > 0. 

2. (n — 1) eigenvalues of [/ — (gh'/h 'g)]F have negative real parts. 

It is also possible to derive time domain asymptotic stability conditions 
applying for the case h'g = 0, but we shall not bother with these here. They 
are actually the restatement of the frequency domain conditions given earlier 
for the case when b n = 0 in Eq. (6.2-15). As earlier remarked, the case h'g < 0 
can be shown to lead to instability (see Problem 6.2-3). 

We now make some brief remarks concerning the system (6.2-12), which 
possesses additional nonlinearity over the system (6.2-6). The equation de- 
scribing chattering mode motion is the same for (6.2-12) as for (6.2-6); there- 
fore, for those initial states of (6.2-12) for which chattering will subsequently 
always occur, asymptotic stability conditions are the same as for (6.2-6). 
It is also not difficult to show that for states of (6.2-12) which are suitably 
close to but not on that part of the switching surface where states are known 
to be asymptotically stable, asymptotic stability also holds. Therefore, condi- 
tions for asymptotic stability of (6.2-12) are identical with conditions for 
asymptotic stability of (6.2-6); the precise set of initial states that define 
asymptotically stable trajectories will, in general, differ for the two systems. 

The question of global asymptotic stability is yet more complicated. 
We discuss it briefly here for the sake of completeness. Note that whereas 
the asymptotic stability results may be applied to optimal systems, the global 
asymptotic stability results may not generally be so applied — the next section 
will discuss these points more fully. 

One way to ensure global asymptotic stability is to require that the fol- 
lowing set of conditions, associated with the name of Popov, (12], be sat- 
isfied. 

1. All eigenvalues of Thave negative real parts. 
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2. There exist nonnegative numbers a and b , not both zero, such that 
R c[(jcoa + b)h'(jcoI — F)~'g]> 0 for all real co, except that if b = 0, 
the inequality sign may be replaced by an equality sign at a> = 0. 

A graphical procedure exists for checking the existence of a and b such that 
the inequality holds. Observe that 

R e[(yo)0 + b)h\jcoI — F) _I g] 

= b R z[K(jcoI — F) -1 g] — acolm[h'(jcol — - F)~ J g]. 

Thus, if a plot is made for varying co with x coordinate Re[h'(jcoI — F)~ 1 g] and 
y coordinate coIm[h'(j(oI — F)~ l g] {as distinct from merely Im[h'(jcoI — F)~ 1 g], 
which would be the case for a Nyquist plot], the inequality condition becomes 
bx — ay > 0 for all points [x, y] on the plot. Therefore, if the plot lies strictly 
below and/or to the right of any straight line of slope between zero and infinity 
(save for a line of slope zero, the plot may and will meet the line at the origin), 
then global asymptotic stability is established. 

The regular Nyquist plot of h'(ja>I — F)~ x g can be used to check the 
inequality condition if either a = 0 or b = 0, as inspection of the inequality 
will readily show. Problem 6.2-6 asks for a partial proof of this result. With 
b — 0, the result is established in [10] by construction of a Lyapunov function. 



Example. Consider the transfer function ( s + 3 )/(s + l)(y + 2), which has the 
state-space realization 







u. 



y — [ 3 l]x. 

Suppose that a relay is inserted in a feedback loop around a plant with these 
equations. Observe that Kg = 1, guaranteeing by (6.2-8) that endpoints occur. 
From (6.2-7), we conclude that endpoints are defined by 

3*i + * 2 = 0, “1 < -2x x < 1. 

This is a finite segment, with the origin as midpoint, of the straight line 3x x 

+ Xi = 0 . 

Examination of the frequency domain conditions for stability shows that 
the sufficiency conditions are fulfilled, since the transfer function numerator, 

s + 3, has degree exactly one less than that of the denominator, and has no 
nonnegative real part zero. 

The equation of motion of the chattering regime is 

~° n 

X. 

LO -3J 

The eigenvalues of 

“0 r 

-0 — 3_ 
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are zero and —3, which again implies stability through the time-domain 
criterion. 

Last, we can consider global asymptotic stability. Taking a = 1, b — 1.5 
leads to 

ReO-o*, + ‘>*' 0 “' - 

from which it is clear that the criteria for global asymptotic stability are ful- 
filled. 

Problem 6.2-1. Given the transfer function of (6.2-15), it is known that the 
associated relay system is asymptotically stable so long as b n > 0 and the zeros of 
b n s n ~ l -f — * + b y have nonpositive real parts. Show that the condition b n > 0 
is equivalent to the condition h'g > 0. Show also that the condition that the zeros 
of b n s n ~ { + ** * have negative real parts is equivalent to the condition that 
( n — 1) eigenvalues off/ — (gh'/h'g)]F have negative real parts. [Hint for second part: 
Write down a state-space realization for sh'{sl — F)~ l g, and for the inverse of this 
transfer function. Study the “F matrix” of the inverse, and compare its properties 
with those of the numerator polynomial of sh'(sl — F)~ s g. This problem illustrates 
the equivalence between part of the frequency domain condition and the time 
domain condition for asymptotic stability.] 

Problem 6.2-2. Show that a necessary and sufficient condition for the 
asymptotic stability of all trajectories of x = [/ — (gh'/h'g)]Fx with initial condition 
h'x o = 0 is that (n — 1) eigenvalues of [/ — (gh'/h'g))F have negative real parts. 

Problem 6.2-3. Show that if h'g < 0, neither of the necessary frequency 
domain conditions for asymptotic stability holds. 

Problem 6.2-4. Suppose that x ~ [I — (gh'/h'g)]Fx has n — 1 eigenvalues 
with negative real parts. By changing the coordinate basis so that the new F matrix 
becomes the direct sum of an (n — 1) x (n — 1) matrix F l and the 1 X 1 zero 
matrix, show that there exist nonnegative definite matrices P such that x'Px ^ 0 
unless x = Xh for some constant X, and such that V(x) = x'Px is a Lyapunov func- 
tion for all initial states x 0 satisfying h'x 0 — 0. 

Problem 6.2-5. Let a P matrix be found as in Problem 6.2-4. Consider the 
tentative Lyapunov function x'Px -f h'x sgn (h'x) for the relay system, as distinct 
from the linear system x = [/ - (gh'/h'g)]Fx. For all x 0 on h'x 0 = 0 and such that 
the system operates in the chattering mode, this tentative Lyapunov function 
becomes x'Px, and it establishes asymptotic stability for these states. Show also that 
for *o suitably small, but not satisfying h'x 0 = 0, asymptotic stability still prevails. 
(Thus it is established that asymptotic stability of the chattering regime alone implies 
asymptotic stability, with no qualification.) 

Problem 6.2-6. Suppose the Popov conditions hold. Except for a pathological 
situation, they can be shown to imply the existence of a positive definite symmetric 
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matrix P and a vector / such that PF + FP = -IV and Pg = bh + aF'b + 
v / 2 aiYg l . Adopt as a Lyapunov function V = -f 2a/fx sgn [A'x], Assume 
h'g > 0, and show that V < 0 off and on the switching surface. (It is then possible 
to show that V cannot be identically zero along a trajectory, but this is a good deal 
harder. Consideration of the pathological situation and the case h'g = 0 can be 
made, and global asymptotic stability can then be established.) 



6.3 INTRODUCTION OF RELAYS INTO NOMINALLY 
OPTIMAL SYSTEMS 



Our purpose here is to indicate how the results of the preceding section, 
particularly those on stability, may be put to use in connection with optimal 
linear systems. Let us assume that a linear system x = Fx + gu , which 
is completely controllable, has associated with it an optimal feedback 
law u = k'^x, resulting from minimizing a performance index of the form 
e 2at (u z + x'Qx ) dt. In this index, a is nonnegative and the usual observa- 
bility requirement is satisfied, guaranteeing that the closed-loop system has 
degree of stability a. Assume now that the resulting optimal system is modified 
by the introduction of the relay as indicated in Fig. 6.3-1. 




Fig. 6.3-1 Optimal system with relay added in loop. 



To carry out an analysis of the system of Fig. 6.3-1, we need to make use 
of the two equations defining the optimal performance index x'P a x and 
optimal control law u = k r a x. These are 

P,(F + a/) + (F' + a I)P. - P x gg'P a + 0=0 (6.3-1) 

and 

K = -Pag- (6.3-2) 

Recall, too, that P x is that particular solution of X(F + a/) + (F' + aI)X 
— Xgg'X +0=0 which is positive definite. 

We shall consider first the asymptotic stability of the scheme of Fig. 
6.3-1, using frequency domain and then time domain ideas, and subsequently 
make some brief remarks concerning the global asymptotic stability of the 
scheme. We shall not consider multiple-input systems or time-varying sys- 
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terns, although certainly the results would be expected to extend to such 
systems. 

Reference to the preceding section shows that the asymptotic stability 
properties can be inferred from properties of the transfer function —k’Jjsl 

— g'P a (s/ — F)~ x g. In particular, we recall that if the degree of the 
numerator of the transfer function is one less than that of the denominator, 
and the leading coefficient of the numerator is positive, a necessary condition 
for asymptotic stability is that the numerator zeros have nonpositive real 
parts, and a sufficient condition for asymptotic stability is that the numerator 
zeros have negative real parts. 

We shall now check that the numerator of g'P a {sl — F)~ l g has degree 
precisely one less than the denominator, with positive leading coefficient. 
Since the denominator of this transfer function has degree n, it follows that 
the coefficient of r"" 1 in the numerator of the transfer function is lim sg'P a (sI 

— FY l g. (If this were zero, there would be no term involving in the 
numerator.) Now, 



sg'P a (sI ~ F)~ l g - g'P a (sI - F)(sl - FY'g + g'P a F(sI - Fy l g 



Therefore, 



- g’P«g + g'P a F(sl - F)~ l g. 



lim sgT^sl - F)' l g - g'P a g, (6.3-3) 



which is positive because P a is positive definite. 

What of the zeros of —k^sl — F)~ 1 gl We recall that the closed-loop 
optimal system (no relay being present) retains degree of stability a even if a 
gain constant is introduced in the feedback loop, for any /? in the range 
i As approaches infinity, those poles of the closed-loop system 

that remain finite must approach the zeros of —k f a (sl — F)~'g , by root locus 
theory. Now for /? < oo, the closed-loop poles must lie to the left of Re|VJ 
= —a; therefore, we conclude that the zeros of ~k’ a {sl — F) l g must lie to 
the left of Re[s] = — a, or on this line . 

Consequently, if a = 0, the zeros of —k' a (sl — F)~ l g have nonpositive 
real parts, and the necessary (but not sufficient) condition for asymptotic 
stability is fulfilled, whereas if a > 0, the zeros of —k' a (sl ~ F)~ l g have 

negative real parts, and the sufficient condition for asymptotic stability is 
fulfilled. 

It may be thought that, for the case a = 0, the zeros of — k' a (sl — F)~ l g 
should have negative, as distinct from nonpositive, real parts. That this 
need not be so is established from a simple example: 





~ 0 “ 

1 ’ 




'0 0 
0 1 




Sec. 6.3 INTRODUCTION OF 



RELAYS INTO NOMINALLY OPTIMAL SYSTEMS 113 



This case leads to 

r ° 

*’"U - VT_’ 

and to a transfer function —k'^sl — F)~ { g, which has a zero for s = 0. 

We shall now take a time domain approach to discussing the stability 
of the arrangement of Fig. 6.3-1. In particular, we shall exhibit a Lyapunov 
function. From the material in the previous section, we know that there will 
be a chattering regime, since —k^g = g'P a g > 0, and that the motion of the 
system in the chattering regime is governed by 

x = |~/ - Fx. (6.3-4) 

L g P*g\ 

For motion on and off the switching surface g'P a x = 0, we adopt as a 
Lyapunov function 

V(x) = x'P a x + g f P a x sgn ( g'P a x ). (6.3-5) 

Observe that V is a continuous function of x, on and off the switching surface. 
It turns out that the function V has sufficient differentiability properties to 
allow evaluation of V except at isolated and therefore insignificant points, 
located where the system changes from the nonchattering to the chattering 
mode, or vice versa. 

First, off the switching surface, we have 

x = Fx — g sgn (g7>.x), (6.3-6) 

and so 

V = x\P a F + F'PJx — 2x'P u g sgn (g'P a x) + g'P a Fx sgn (g'P a x) 

~ g'Pag [ sgn (g'P«x)] 2 - 
Now use (6.3-1) to obtain 

V - ~x'(2ocP a + Q- P A gg'P a )x - x'(F’P a g + 2 P a g) sgn (g'P a x) - g'P a g. 

(6.3-7) 

That is, V is the sum of terms that are quadratic in x, linear in x, and constant. 
The constant term is always negative; therefore, for suitably small x, V < 0. 
Moreover, since V — >■ 0 as x'x 0, the ratio —V/V increases without bound, 
implying that the instantaneous degree of stability becomes unbounded. On 
the switching surface, however, a different situation occurs. First, g'P^x is 
identically zero, so that, from (6.3-5), V(x) is simply x'P^x. Using (6.3-4), 
we have 



V(x) - x \P a F + F'P a )x 



— F + F' 

V g Pag g P a g / 



But since g'P a x is zero, the second term disappears. Then, using (6.3-1) and 
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expanding the first term, we obtain 

V(x) = —2ax'P x x — x'Qx + x'P^gg'P^x. 

Again, the last term is zero, and so 

K(x) — — 2ccx'P a x - x'Qx. (6.3-8) 

For a > 0, asymptotic stability is immediate. Indeed, since now V is simply 
x'P a x, Eq. (6.3-8) implies V/V < — 2a , so that there is a degree of stability 
equal to a. This is to be expected, because the nonzero eigenvalues of 
[/ __ (gg / P a /g'7 > a g)]F(which determine the stability properties in the chattering 
regime) are none other than the zeros of the numerator polynomial of — k f a (sl 
-F) 1 g, which have been shown to have real part less than or equal to —a. 

The techniques of the previous chapter also allow the system of Fig. 
6.3-2 to be handled, where j? is a time-varying nonlinearity with the usual 
constraints — viz., j?(I, t) > > 0 and /?(— 1, /) < /? 2 < 0 for all t together 

with continuity and boundedness of /?(1, t) and /?(— 1 , /). Conditions for 
asymptotic stability of the system of Figs. 6.3-1 and 6.3-2 are the same. It 
also turns out that the Lyapunov function of (6.3-5), used in connection with 
the system of Fig. 6.3-1, serves as a Lyapunov function for the system of 
Fig. 6.3-2. Verification of this fact is sought in Problem 6.3-3. 




Fig. 6.3-2 Relay and nonlinearity associated with optimal system. 



In the first section of this chapter, we demonstrated stability properties 
for nominally optimal regulators that had input transducer nonlinearities. 
We did not establish stability of course for arbitrary nonlinearities. However, 
the insertion of a relay does serve to guarantee asymptotic stability for an 
almost arbitrary class of input nonlinearities. Therefore, there may be 
advantages in using a relay other than for preventing saturation. What inser- 
tion of the relay unfortunately does not guarantee is global asymptotic sta- 
bility. Thus, the use of a relay for stabilizing purposes must be with caution. 

As remarked earlier, it is not possible to draw general conclusions 
about the global asymptotic stability of optimal linear systems into which a 
relay has been inserted, and the Popov criterion of the previous section seems 
to be the only helpful tool. There is, though, one class of transfer functions 
k*(sl F) l g that will often arise for which global asymptotic stability is 
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Fig. 6.3-3 Nyquist plot from which global asymptotic stability 
for system with relay could be concluded. 

an immediate consequence of the Popov criterion. This is the class for which 
Fis asymptotically stable and the Nyquist plot of w(jco) = — k' a (ja>I — F)~ x g 
avoids, for positive co, the half-plane Im[w(ycu)] > 0 (see Fig. 6.3-3). In this 
instance. 



-~co lm[—k' a (jcoI — F) l g] > 0 

for all real co, except for co = 0, because for positive co, the Nyquist plot prop- 
erty just described guarantees Im[vv(/a>)] < 0, except for co = 0. The Popov 
criterion conditions are then satisfied. 

Problem 6.3-1 . Consider the system of Fig. 6.3-1 , and suppose that a constant 
gain /? is inserted in the feedback part of the loop. Show that the equation of the 
chattering regime is unaltered. Can you prove that the equation also remains 
unaltered with time-varying and nonlinear /?? Discuss the stability properties of 
the closed-loop system with the aid of Lyapunov functions. Is it possible to con- 
clude that V < 0 for a larger or smaller set of x when variation from unity feedback 
is introduced? 



Problem 6.3-2. Sketch the graph of the describing function of a relay (viz., 
the graph of the amplitude of the fundamental component of the relay output with 
relay input A sin cot , versus A). Discuss why this graph suggests asymptotic 
stability for sufficiently small initial conditions of the closed-loop system of Fig. 
6.3-1. 

Problem 6.3-3. With notation as defined in this section, show that a Lyapunov 
function for the system of Fig. 6.3-2 is provided by x'P a x + g'P a x sgn (, g'P a x ). 
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CHAPTER 



SENSITIVITY ASPECTS 
OF OPTIMAL REGULATORS 



7.1 THE SENSITIVITY PROBLEM 

In this section, we shall introduce the so-called sensitivity problem and 
present a number of background results. The results are then applied in the 
later sections of this chapter to develop further properties and techniques asso- 
ciated with the optimal regulator. 

One of the original motivations for placing feedback around systems is 
to reduce their sensitivity to parameter variations. Thus, consider the arrange- 
ments of Fig. 7.1-1. Both schemes provide the same nominal gain, viz., 
A/(\ + Ap). But now if the block with gain A suffers a gain change of 1 %, 
the closed-loop scheme undergoes a gain change of approximately 
l/d + Aft) %, whereas the open-loop scheme undergoes a gain change of 
1 %. Consequently, if 1 + Ap is made large, the closed-loop scheme is judged 
better than the open-loop scheme from the point of view of sensitivity of 
input-output performance to plant parameter variations [where we think of 
the block of gain A as the plant, and the blocks of gain p and 1/(1 + A/3) 
as controllers]. 

Although this notion applies to the situation where A and p may be 
frequency dependent, there is no straightforward extension to, say, multiple- 
input systems, or situations where the varying plant parameter is not gain but 
something else, such as an initial condition. Therefore, we shall be concerned 
in this section with setting out on a firm basis what we mean by sensitivity, 
and sensitivity reduction, in these more general situations. 
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Fig. 7.1-1 A closed-loop scheme and an open-loop scheme. 

To begin with, we consider the closed-loop and open-loop schemes of 
Fig. 7.1-2. Two ways are illustrated of controlling the plant, which is allowed 
to have multiple inputs and outputs. We also make the following assumptions. 

Assumption 7.1-1. The overall transfer matrices of the two schemes — viz., 
[/ + and jP(s)C(s) — are the same; 

Assumption 7.1-2. The controlled systems are asymptotically stable. {In 
particular, [/ + P(jco)F(jco)]~ 1 must exist for all real CO.} 

The effect of these two assumptions is to guarantee that the effect of 
initial conditions on the output will die out. When this has occurred, identical 
inputs to the two schemes will produce identical outputs, permitting the per- 




Fig, 7.1 -2 Feedback compensator and series compensator. 
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formance of the two schemes to be meaningfully compared. As will be shown 
subsequently, the return difference matrix I + P(jat)F(jo)), so -called by 
analogy with the corresponding scalar quantity, will play a key role in quanti- 
tatively making the comparison. 

We now permit the possibility of parameter variations in the plant. 
This means that the transfer function matrix of the plant must be regarded 
as a function of a parameter //, and that ji can vary away from a nominal value 
/x nom . Plant parameter variations may also be of the type where that part of 
the output due to plant initial conditions (and this part alone) may be affected. 
This would be the case if, e.g., the plant parameter in question was actually an 
initial condition, or if some states of the plant were uncontrollable but observ- 
able, and the way these uncontrollable states were coupled to the output was 
varied. Also, both the transfer function matrix and the initial condition part of 
the output may be parameter dependent. Accordingly, we shall write 

Y(s; fi) = P(s; fi)U(s ; n) + Z(s; //). (7.1-1) 

In this equation, which applies to both the open- and closed-loop arrange- 
ments, Y(s\ ju) is the (Laplace transform of the) output of the plant, P(s; ju) 
is the plant transfer function matrix, Z{s\ //) is that component of the plant 
output due to initial conditions, and U(s; fx) is the input to the plant {not 
the input to the whole controlled system). Notice that, at least for closed- 
loop control, U(s ; fx) depends on ix because it partially depends on the feed- 
back output. Normally, jj = // nom , the nominal value of ju. 

Now we aim to compare the two schemes of Fig. 7.1-2. Accordingly, 
assume that the initial conditions on the plant, as well as the external inputs 
R(s ), are the same for both cases. In other words, if the parameter is at its 
nominal value, the input U(s; fi nom ) to the plant itself is the same for both 
schemes. When the plant parameter changes, we shall assume it changes in 
the same way for both plants; accordingly, the variation in the plant output 
Y(s; fx) due to initial conditions will be the same in both cases. But the 
variation in y^; ff) due to the plant inputs will be different. Certainly, 
P(s; ff) will change in the same way for both schemes. For the open-loop 
scheme, U(s ; ff) will be unchanged from U(s ; /x nom ); however, for the closed- 
loop scheme, that component of U(s; ff) due to R(s) will remain unchanged 
from the corresponding component of U{s\ jx nom ), but that component due 
to feedback of the plant output Y(s\ ju) will be changed. Thus, there is an 
overall change in the plant input, which hopefully will compensate for para- 
meter variation. 

We now consider these notions quantitatively. Let subscripts o and c 
refer to the open- and closed-loop schemes, respectively, and to keep the 
notation as simple as possible, let us omit the Laplace transform variable for 
the next few equations. We shall be deriving a fundamental equation, (7.1-4), 
that compares the variation caused in the output by a plant parameter change 
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for the open- and closed-loop control arrangements. Before we can compare 
these variations, Y 0 (v) - Y 0 {/u nom ) for the open-loop arrangement and Y//z) 
— Y c (finom) for the closed-loop arrangement, we shall derive separate expres- 
sions for both. With a parameter change from /z mon to an arbitrary //, the 
external input R is, of course, unaltered. Also, at the nominal parameter value, 
the plant inputs £/ c (/z nom ) and U 0 (/z aom ) must be the same — likewise for their 
outputs. 

For the open-loop control scheme, so long as R remains the same when 
the parameter is varied, we have U 0 (/i nom ) = L/ 0 (/z). Thus, from Eq. (7.1-1), 

Y 0 (m) - YJfi riom ) = - P(M nom )]UJfi no J + Z(n) - Z( Maam ). 

(7.1-2) 

As remarked previously, when /j, is varied, the input to the plant for the 
closed-loop control scheme is varied, even though R is left unaltered. Thus, 
(7.1-1) yields 

Y c (ju) - Y c (jln om) = P(JU)U C (M) ~ P(MnomWc(Mnom) + Z({l) - Z(// nom ). 
We can, however, eliminate U c (n). Reference to Fig. 7.1-2 shows that 

u c GO = -FY'Qi) + R 

= ~F[Y C (R) ~ Y e Ql m )] ~ FY c (Mnom) + R 

= -F[Yc(M) ~ Y c(M nom)] + U c (jU nom ) 

where we have implicitly used the fact that R , the controlled system input, is 
the same for two values of /z. Using this expression for U c (f)> we have 

Y c (M) ~ Y c (Mnom) - ~P(ju)F[Y c (m) - Y c (m»o J] 

+ [P(M) ~ P(M a om)]U c (fi no m) 

+ Z(/Z) - Z(/z no J 

or 

[/ + P(r)F][Y Xf) — Y c (/Z n ora)] 

= [ W - A/*nom)R(/AoJ + Z(/z) ~ ZOO. (7.1-3) 

Equations (7. 1 -2) and (7. 1 -3) are the two expressions for the output variations 
resulting from plant parameter variation from /z nom to //. They are, perhaps, 
of some independent interest for design purposes, but the connection between 
them is of greater importance. Recalling that U c (/z nom ) and £/ 0 (/z nom ), the two 
plant inputs corresponding to the two control configurations, are the same, 
it follows immediately from (7.1-2) and (7.1-3) that 

yds; H) - Y c (s; fi m ) = 11 + P(s ; M ) - Y 0 (s; 

( 7 . 1 - 4 ) 

Equation (7.1-4) relates the variations in the open- and closed-loop plant 
outputs. We recall it is valid for the following: 
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1. All system inputs R{s), i.e., the relation is independent of the par- 
ticular input used, so long as the same input is applied to the open- 
and closed-loop systems. 

2. Any form of plant parameter variations. This includes variation of 
the transfer function matrix, initial condition response, multiple 
parameter variations, or any combination of these. 

Of course, the plant and controller transfer function matrices do not have 
to consist of rational functions of j. 

If the variation from p nom to p is small, and if Y 0 (s; /*), Y c (s; p)> and 
P(s; jj) depend continuously on //, Eq. (7.1-4) may be replaced by the approxi- 
mate equation 

Y c (s ; n) - y/j; // nom ) = [/ + P(s; Mnom)F(s)]~ l [Yo(si M) ~ Y <>(*', Aom)], 

(7.1-5) 

where the variations of outputs are related by a formula involving the nominal 
rather than the perturbed value of the parameter. This equation may there- 
fore be more useful, even though it is an approximation. 

As another interpretation of (7.1-5), we can also define sensitivity func- 
tions aft) and gJj) associated with the closed- and open-loop outputs. These 
are given by 

°c«) = (yXt))} Ojit) = \l (y,( 0)1 (7.1-6) 

assuming the differentials exist. Then Eq. (7.1-5) implies the following rela- 
tion between the Laplace transforms E c (*y) and S 0 (^) of the closed- and open- 
loop sensitivity functions: 

2 c(s) - V + P(s; MnoJF(s)]-'Xo(s)- (7.1-7) 

For the remainder of this section, we shall discuss the question of charac- 
terizing reduction , via feedback, of sensitivity to plant parameter variations. 
We shall first state a criterion for a closed-loop scheme to be better than an 
open-loop one in terms of variations caused in the plant output of each 
scheme by a parameter variation. Then we shall interpret this criterion as a 
constraint on the return difference matrix of the feedback arrangement, 
with the aid of Eq. (7.1-5). 

Suppose that variation of a parameter from the nominal value p aom to 
/ 1 = Rnom + A/* nora , where Ap nom is small, causes a change in the closed-loop 
system output of y c (t\ p) — y c (t\ p nom ) and in the open-loop system output of 

y 0 U> M) ~ y 0 Ul Mnom)- We say that the closed-loop scheme is superior to the 
open-loop scheme if for all t t and all y 0 (- ; ju aom ), 

j ' [?,(*; p) - y£c, ju nom )]'[.VcO; v) - y c (f, dt 

J to 

< f ‘ [y 0 (p, m) - yjj-, /0]'LK,(f; n) - y 0 (t; Hn™)] dt. (7.1-8) 

J to 
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(Here, t Q is an arbitrary initial time.) Equation (7.1-8) is equivalent to requir- 
ing that the integral squared error for the closed-loop scheme never exceeds 
that for the open-loop scheme. 

By use of Parseval’s theorem and several technical artifices, it is possible 
to show that (7.1-8) implies 

f [Y c (jco; n) - Y c (jco; n D0 J}*'[Y c {j(O', u) - Y c (jco; fi ao J] dco 

J — OO 

< [ {YJjor, n) - Y a (jco; M a0 m)]*' [Y„(jco ; ft) - YJjco ; dco. 

Now use Eq. (7.1-5) [and recall that [/ + P(jco; p)F(jcoi)]' 1 has been assumed 
to exist, to guarantee asymptotic stability of the closed-loop scheme}. We 
obtain, then, 




[K 0 (/a>; //) — // nora )]*'[7 — 



S*'(j<o)S(ja>)} 



[Y 0 (jco; m) ~ Yfjco; ft nom )] dco> 0 (7.1-9) 

where 

S(s) ^ [I + P(s; // qo JF(*)]->. (7.1-10) 

Observe that S(.y) is the inverse of the return difference matrix. A sufficient 
condition for (7.1-9) to always hold is thus 



I - S*'(j(o)S(jco) > 0. (7.1-1 1) 

Equivalently, with T(s) = I + P(s; fi aom )F(s ) (the return difference matrix), 
we have 

T*'(jco)T(jco) - I > 0. (7.1-12) 

Consequently , (7.1-1 1) and (7.1-12) constitute sufficient conditions for sensitivity 
reduction using feedback . 

Actually, (7.1-11) and (7.1-12) would be necessary if the variation 
Yfjco; n) ~ Yfjco; ju aom ) were an arbitrary Fourier transform. But it is the 
Fourier transform of yft; //) - y 0 (t; /i oom ) 9 which is not in general an arbi- 
trary function. Hence, strictly speaking, (7.1-11) and (7.1-12) are not neces- 
sary. Apparently, though, the sorts of variation that arise are such as to make 
these equations “almost necessary/’ and one could safely rule out the 
existence of any other condition on the return difference matrix for sensitivity 
reduction in most cases. 

In the scalar input case, the classical result 



\T(jco)\> 1 (7.1-13) 

is recovered. This is the result we noted at the start of the section, but this 
derivation has, of course, given the constraint (7.1-13) a much wider inter- 
pretation than previously. 
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There is some validity in the concept of degree of sensitivity improvement 
(as distinct from sensitivity improvement itself) associated with equivalent 
closed- and open-loop systems. Thus, consider two control systems, each 
being realized in both closed- and open-loop form. We could say that system 
1 offered a greater degree of sensitivity improvement than system 2, if the 
inequality 

r fj) ~ y c i0; /<no«)]ki(^ m) - y c \if\ ^nom)] dt 

J h 

rti 

< tj'oiO; m) - y 0 10; m) - y „ »(*; /'..J dt 

J to 

were in some way “stronger” than the inequality 

f ' bczO; m) - y C i(c, /*.».»)]' u) - jM*; AjJ] dt 

J to 

< f ' [y 0 i(f, m) - y oi (t; fin a J}[yoi(t\ u) - y oi (t; y“ aom )] dt. 

J to 

Equivalently, this would require the inequality 

/ - Sr(jco)S t Uco) > 0 

to be “stronger” than the inequality 

/ - Sr(jco)S 2 (jco) > 0 . 

If S x and S 2 are of the same dimension, it is clear that the rigorous inter- 
pretation of this statement regarding the “strength” of the two inequalities is 



nr 


I - srU(o)S\(jco) > / sr(jco)S 2 (jco) 




vl 


S*'(jco)S 2 (jco) > S*'(jco)S,(jco). 


(7.1-14) 


Equivalently, one can require 






Tr(jco)T,(joj) > T*\jco)T 2 U(o), 


(7.1-15) 


which certainly accords with the well-known ideas pertaining 
systems. For the latter, (7.1-15) becomes 


to scalar 




\T,(jco)\>\T 2 (jco)\. 


(7.1-16) 



The interpretation is that larger return differences give greater reduction in 
sensitivity than smaller return differences. 

The results of this section stem from work by Perkins and Cruz, amongst 
whose papers we note [1]. The approach taken here is closely allied to that 
of Kreindler [2]. It is also interesting to note that all the ideas may be extended 
to time-varying systems [3]. 

Besides the property (7.1-11) asked of S(s)— the inverse of the return 
difference matrix, to ensure sensitivity improvement via feedback, we recall 
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Assumption 7.1-2, which implies that S(s) = [I + P(s; fi Rom )F(s)]~ l should 
have analytic . elements in R e[s] >0. Square matrices possessing these two 
properties are termed passive scattering matrices in the network theory 
literature, where they play a prominent role — see e.g., [4]. 



Example. Consider a plant with transfer function 1 /(s -f l)(s 4- 10) and sup- 
pose a feedback controller of transfer function a constant, K, is used. The 
feedback is negative. 

The return difference is the loop gain plus unity, or 



It follows that 



njco) i + + i xyct) + ioj 

_ -CO 2 + (10 + K) + ll/co 
-CO 2 + 10 + llyco 



I T(ja>)\ 2 - 1 



X(-2co 2 + 20 + a:) 

( — co 2 + 10) 2 + 121co 2 



We conclude that for no value of K will this quantity always be nonnega- 
tive. But for K = 1, say, which ensures that the closed-loop system will be 
asymptotically stable, the above quantity will be nonnegative for co < a/\U5. 
Consequently, if interest centers around performance of the system only up to 
a frequency of a/T 03 rad/sec., we achieve a reduction in sensitivity to plant 
parameter variation, i.e. if one of the plant poles were to vary, or the plant gain 
to vary, the presence of the feedback would reduce the effect of this variation. 

Above 10.5 rad/sec., we would however expect to see a heightened rather 
than reduced sensitivity to plant parameter variations. 



Problem 7.1-1. With reference to Fig. 7.1-1, compare the output sensitivity 
to variation in the /? block of the closed -loop system, and the 1/(1 + Afi) block of 
the open-loop system; then, for Fig. 7.1-2, compare the output sensitivity to varia- 
tion in the F(s) block of the closed-loop system, and the C(s) block of the_ open -loop 
system. 




Fig. 7.1-3 Combined series compensator and feedback com- 
pensator. 



Problem 7.1-2. Consider the arrangement of Fig. 7.1-3, where a series com- 
pensator and a feedback control are used. What is the criterion for this system to 
offer a sensitivity improvement (for plant parameter variations) over an equivalent 
open-loop system? 
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Problem 7.1-3. Consider Fig. 7.1-3, with the time-invariant plant and con- 
troller replaced by a linear time-varying plant and controller, of impulse responses 
p(t y t) and /(/, t), respectively. The operator S(t — t) + {'_ „/>(/, A)/(A, r) dl is 
known as a causal Volterra operator, and can frequently be shown to possess a 
causal inverse, s{t y t). Show that 

>’c0; /*) - yjt; H nom) = I ' s{t, t; /t) - y 0 (r; ju Boa )] dr. 

J — oo 

Problem 7.1-4. Consider the arrangements of Fig. 7.1-4, which are intended 
to be the same as those of Fig. 7.1-2, except for the introduction of a disturbance 
of Laplace transform D{s) at the plant output. Show that the integral square error 
in the output due to D(s) for the closed-loop scheme is less than or equal to the 
same quantity associated with the open-loop scheme if the closed-loop scheme offers 

sensitivity improvement over the open-loop scheme of the sort discussed. 

* 




Fig. 7.1-4 Block diagrams showing the introduction of distur- 
bances. 



7.2 SENSITIVITY REDUCTION IN OPTIMAL REGULATORS 

In this section, we shall apply the results of the preceding section to 
optimal regulators. For the moment, we shall regard the optimal regulator 
as consisting of a “plant,” with transfer function —k! a (sl — F)~'g, and unity 
negative feedback. The output is taken to be — k’ x x, where x is the state of the 
system 

x = Fx + gu 

to which optimal control is applied. The situation is thus that of Fig. 7.2-1. 

With these definitions, the criterion for this closed-loop scheme to have 
a lower sensitivity to variations in F, g, k x or the plant initial condition than 
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Fig. 7.2-1 Regulator arranged so as to have unity feedback. 

an equivalent open-loop system (obtained by cascading a suitable controller 
with the plant) is the following inequality on the return difference, derived in 
the last section: 

|1 ~k:UcoI~F)' i g\> 1 (7.2-1) 

for all real o>. 

The fact that the regulator is optimal ensures that this inequality should 
hold, which is immediate from the fundamental Eq. (5.2-12) of Chapter 5. 
Thus, we conclude that, with the appropriate definitions of plant and output, 
optimality implies improvement in sensitivity over an equivalent open-loop 
system. 

We shall now derive a corresponding result for multiple-input systems. 
We recall the fundamental equation (5.2-7) of Chapter 5: 

[/— R U2 K' 0 (jcoI — F)~ i GR~ i/2 ]*'[I — R in K ! 0 (ja>I — F)~ l GR~ Uz ] > /. 

(7.2-2) 

With K 0 replaced by K a , the inequality is also valid. We may regard (7.2-2) 
(with replacing K 0 ) as stating that 

T*Xja>)T(ja>) - / > 0 (7.2-3) 

where 

T(jco) = / - R V2 K f u (jo)I - F)-'GR~ U \ (7.2-4) 

We now examine the implications of this inequality using Fig. 7.2-2. 
The three diagrams in this figure all show equivalent systems, the first showing 
how the transfer function matrix of the system is achieved by unity negative 
feedback, the second linking the first and the third, and the third showing 
how the transfer function matrix is achieved using memoryless transforma- 
tions of both the input and the output, with a unity negative feedback 
arrangement linking the transformed input and output. Figure 7.2-3 shows 
this unity negative feedback part in isolation. 

The return difference matrix for the arrangement of Fig. 7.2-3 is precisely 
the preceding matrix T(jco). Consequently, the feedback arrangement of Fig. 
7.2-3 has lower sensitivity to plant parameter variations than an equivalent 
open-loop system. 

There is also a simple consequence following from the equivalences of 
Fig. 7.2-2, which is relevant for the first arrangement shown in this figure. 
We retain the notation of the previous section and introduce the new variable 
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Fig. 7.2-2 Equivalent systems. 




z, defined by z — R 1/2 y. The sensitivity reduction associated with the 
gement of Fig. 7.2-3 implies that 

rti 

I [z c (M) ^c(/^nom] [z e (M) ^£fl nora)] dt 

J to 

rti 

< [Zo(M) - Z o(H ,om)n^W - *o(M noJ] & 

J t o 

A 

for all t o, t, and variations in z 0 . Since >> = R^ l/1 z i it follows that 

J bcOO - y£Vnom)]'R[y£v) - y£t* M J] dt 

J to 

< f ‘ [y,(M) - y<,(ftnom)]'R[y 0 (M) - y 0 (Mn 0 m)] dt. 

J to 



arran- 





128 SENSITIVITY ASPECTS OF OPTIMAL REGULATORS 



Ch . 7 



This is certainly meaningful. Suppose we merely replaced the original defini- 
tion of reduction of sensitivity, viz., 

[ ' [yX/i) - - y c (ft na J] dt 

J fo 

< f 1 bX/O - jWfnoJHjWO - yXMnJfl dt 

J to 

for all t Q , t { and variations in y oy by one where we can weight the various 
entries of the error vectors separately. Namely, 

f ’ Mm) - y e <ji*oJY w[y c (ju) - y c (M* 0 Jl dt 
J 1 0 

< f ' h’o (a) - y 0 (jinom)]'W[y 0 (ji) - y 0 (ju aom )\ dt 

J to 

for some positive definite W 9 all r 0 , t , and variations in y 0 . We may then con- 
clude that the first scheme of Fig. 7.2-2 offers a sensitivity reduction over an 
equivalent open-loop scheme — i.e., with appropriate definition of plant and 
output, the multiple input regulator is a feedback scheme where plant pa- 
rameter sensitivity is reduced. 

At this stage, we might examine the usefulness of what we have estab- 
lished. Figure 7.2-4 shows how we may expect to use an optimal regulator. 

The true output of the regulator is not — at least, normally K' a x, but rather 

H'x . Now, the above remarks have established that variations in F, G and 
K a give rise to reduced sensitivity in K' a x rather than H'x; consequently, 
we do not have a direct result concerning reduction in sensitivity of the reg- 
ulator output. 





Fig. 7.2-4 Regulator diagram. 

To be sure, it is useful to know that the sensitivity of K^x is reduced, in 
the sense that (7.2-5), rewritten as follows, 

f Mm) — Xc(Mnom)]'KRK[X c (M) — XciMnom)] dt 

J to 

< r W/<) ~ x 0 Ui aom )]'K a RK' a [x o (u) - *X^oJ] dt, (7.2-6) 

J to 

should hold for all t 0 , and variations in x 0 . If, in fact, K'^RK^ were positive 
definite (requiring the number of inputs to at least equal the number of states). 
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(7.2-6) would actually imply that the states themselves have the sensitivity 
reduction property. But, in general, K a RK' a is merely nonnegative definite, 
and such a statement will not normally be true. Nor, of course, will (7.2-6) 
imply a relation like 

P - *c<Xom )]'HWH'[X c (jl) ~ X c GUnora)] * 

J h 

< f [x 0 Qi) ~ xX/i w JYHWH'[x 0 ({i) - x 0 (ji m )] dt 

J to 

for some positive definite W, all t 0 , t 19 and variations in x 0 . 

There is, however, one sensitivity result involving the states x directly 
(rather than K' a x ), which applies for single-input systems, when there are 
restrictions placed on the type of parameter variations. We shall now derive 
this result. 

We consider the arrangement of Fig. 7.2-5 with u restricted to being a 
scalar; for the purposes of applying the sensitivity analysis of the last section, 
we note that the “plant” has transfer function matrix (si — F)~ l g. The quan- 
tities we called Y 0 (s; MaomX Y c (s; //), etc., in the last section will be denoted 
here by X Q (s\ // nom ), A r c (s; //), etc., since in this case the plant output vector 
is actually a state vector. 



Fig. 7.2-5 Plant with state vector as 
output. 




The parameter variations allowed are those where the matrix Fis replaced 
by F + gV for some vector /, all of whose entries are small. We shall show that 
with this restricted parameter variation, the sensitivity of the states is improved 
by the optimal feedback. 

Before doing so, we indicate other types of parameter variation equiv- 
alent to that stated. (Thus, the original restriction need not appear quite so 
narrow.) First, instead of changing F to F + gl\ we may change —k ( a to 
~k\ — rri , where the entries of the vector m are small. This parameter varia- 
tion is equivalent to one where F is changed to F + giri and — k' a is left 
unaltered (see Fig. 7.2-6, which illustrates the equivalence). Second, instead 
of changing F to F + gl\ we may change ~k' a to —(1 + y)k!^ where y 
is a small number. In so doing, we are altering the whole loop gain slightly, 
by the same factor for all frequencies. This is clearly a special case of the pre- 
ceding first equivalence, since -(1 + y)k' a = — k * - m‘ with m = yk a . 
With y small, the entries of m are small. Therefore, this sort of parameter 
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Fig. 7.2-6 Equivalence of certain parameter variations. 

variation is also within the ambit of that originally allowed. It is also easy to 
argue that a variation in g, from g to (1 + y)g where y is very small, is equiv- 
alent to a parameter variation of the original type. 

We now show that, with the sorts of parameter variation allowed, we 
obtain the desired sensitivity reduction. We recall first Eqs. (7.1-2) and (7.1-3) 
of the last section in the original notation, and repeat them for convenience: 

r„(/0 Y Q (ti ao J = [P(n) 7 5 (/z nom )]{/ 0 (// nc , ro ) + Z(ju) — Z(/i nom ) 

and 

[I + p(n)F){Y(p) - y(An om )] 

= [TO - P{H aom )\U c {n nom ) + Z(fi) - 
with, also, U 0 (fi nom ) = U c {/* nom ). 
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Making the various applicable specializations, we get 

X 0 (s ; H) — = [(si — F — gl’Y'g — {si — F)~'g]U 0 {s\ Unom) 

(7.2-7) 

and 

[I — (si — F — gl’)-' gk' a ][X c (s; ix) - X£s; /x n0 J] 

= [(si -F- gl'Y'g - (si - FY'g]U 0 (s; MnoJ- (7.2-8) 
We shall later establish the two identities 

(si- F- gl’Y'g - (si — FY'g = [l’(sl - FY'g)(sI -F- gl’Y'g 

(7.2-9) 

and 

[I — (si — F — gl’Y'gKY'isI -F- gl’Y'g 

= [1 - k’jsl - F - gl'Y'gV'(sI -F- gl’Y'g. (7.2-10) 
Inserting (7.2-9) into (7.2-7) and (7.2-8), we obtain 

X 0 (s', n) - X 0 (s; jx nom ) = [l’ (si - F)-'g](sI - F— gl’Y'gU 0 (s; UnoJ 

(7.2-11) 

and 



X c (s; ix) — X c (s ; = [I’ (si — F) 1 g][I — (si — F — gl’) l gk{] 1 

X (si — F — gl’Y'U 0 (s ; Maom) (7.2-12) 

and inserting (7.2-10) into (7.2-12), we have 
X c (s; n) - X c (s; ^ nom ) 

= [1 - A4(^/- F - gl’Y'g)- '(l’ (si - F)-'g](si - F - gl’Y'gUXs; A„oJ 
= (1 - k’jsl - F — gl’Y 1 gY'[X 0 (s; ju) - X 0 (s; jx no J] 

on using (7.2-1 1). 

So far, we have not insisted that the entries of / be small; all equations 
have been exact. But now, by requiring that the entries of / be small, we can 
make the same sort of approximation as in the last section to conclude that 
(approximately), 

x c (s; IX) ~ X c (s; Mnom) = [1 - K(sl - F)-'g]-'[XAs; m) ~ X a (s; /t nom )]. 

(7.2-13) 

We recognize 1 — k' a (sl — F)~ l g again to be the return difference. 
Thus, (7.2-13) guarantees that for each entry of the state vector (and thus 
for the whole state vector) the closed-loop system provides a lower sensitivity 
to parameter variation of the sort specified than an equivalent open-loop 
system. 

Before commenting further, we establish the two matrix identities used 
previously. First, we deal with (7.2-9): 
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(si- F-gl'Y' -( sI-F )-» 

— (si — F — gl'Y l [(sI - F) - (si- F- gl')](sl - F)-> 

= (. si - F - gl’)-' gl'(sl - FY l . 

Equation (7.2-9) follows by multiplying in the right by g. Second, for (7.2-10), 
we have 

[I — (si — F — gl')~ l gk:](sl -F- gl’Y'g 

— (si — F — gl'Y'g — (si — F — gl'Y'gl K.(sl -F- gl’Y'g ] 

= [1 - K(sl -F- gl'Y'g](si - F - gl'Y'g. 

Equation (7.2-10) follows immediately. 

Because of the fact that each entry of the state vector undergoes the 
sensitivity reduction we have discussed, it follows that if y — Wx is the actual 
output of the regulator with feedback, the sensitivity reduction property of 
the closed-loop scheme extends to y . Quantitatively, multiplication of (7.2-13) 
on the left by h' yields 

Y c (s; M) - Y c (s; ju» o») - [1 ~ k' a (sl - Fy'gY'[Y 0 (s; fi) - Y 0 (s ; // aom )]. 

(7.2-14) 

We stress that y is of more general form here than in, say, Eq. (7.2-5), where 
we had y = —k f a x. We remark again that (7.2-14) is only valid for single- 
input systems, with the restricted type of parameter variations discussed 
previously. 

We round off the theory by discussing the degree of sensitivity improve- 
ment obtained by varying the parameter a in the regulator design, with the 
F , G 9 Qy and R matrices remaining invariant; each a gives rise to a control 
law K^y and we wish to show that different values of a will give rise to differ- 
ent amounts of sensitivity improvement, in the sense discussed at the end of 
the previous section. To see this, we recall first the fundamental relation 
(5.2-6) of Chapter 5, repeated here for convenience: 

[/- F'y i K 0 R {/2 ][! - R l/2 K' 0 (jcoI — F)' { GR~ 1/2 ] 

= / + R- 1/2 G'(-jcoI - F'y'Q(jcoI- Fy i GR~ uz . (7.2-15) 

The corresponding equation when K Q is replaced by K a , obtained as described 
in Chapter 5, Sec. 5.2 is 

[/- R' l/2 G\—ja)I — F'y'K a R V2 ][I — R U2 K f Sja>I — F)~ l GR~ U2 ] 

= / + R~ l/2 G'(~jcoI - F’y i Q(jcoI - Fy i GR~ 1/2 
+ R- l/2 G'(-jcoJ - FT l 2P a (ja>I - F)~ x GR~ l/2 . 

(7.2-16) 

With the definitions 



T x (jco) = / - R U2 K' Q (jcoI - F)~ X GR~ X/Z 
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and 

T 2 (J<q) - / - R' , *K'JUa>l ~ F)- l GR- l/1 , 

it is immediate from (7.2-15) and (7.2-16) that 

Tru^TM^rnmT.uco). ( 7 . 2 - 17 ) 

Therefore, with interpretations of T { and T 2 as matrix return differences in 
the various situations just discussed, we conclude, following the basic theory 
of the last section, that a nonzero a offers a greater degree of sensitivity 
improvement than a zero a. 

An interesting example appearing in [5] applies some of these ideas to a 
pitch-axis control system for a high-performance aircraft. The plant equations 
are third order and linear, but contain parameter variations arising from 
different dynamic pressures. With 

x { = Angle of attack, 

x 2 = Rate of pitch angle, 

x 3 = Incremental elevator angle, and 

u = Control input into the elevator actuator, 

the state equations are 





'-0.074 


1 


-0.012 ' 




0 


X = 


-8.0 


-0.055 


-6.2 


X + 


0 




0 


0 


-6.6667 




6.6667 



for a dynamic pressure of 500, whereas for a dynamic pressure of 22.1, they 
become 





'-0.0016 


1.0 


-0.0002' 




'0 


X = 


-0.1569 


-0.0015 


-0.1131 


x -j- 


0 




0 


0 


-6.6667 




6.6667 



The performance index minimized is J~(w 2 +x'Qx)dt for two different 
matrices Q — viz., 





"10 


0 


O' 




"2 x 10 6 


0 


0 ■ 


Q i = 


0 


0.5 


0 


and Q 2 = 


0 


4 x 10 s 


0 




L 0 


0 


0_ 




L o 


0 


0 



Two optimal control laws u = k\x and u = k' 2 x are calculated, based on the 
first state equation in both cases, and on performance indices including Q l 
and Q 2 , respectively. The control laws are then used not just for the first 
state equation (for which they are optimal) but for the second state equation, 
for which they are not optimal. The following table summarizes the results 
achieved; the sensitivity improvement refers to the sensitivity in x { . 
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State Equation 


Control Law 


Step Response 


Sensitivity Improvement 


First 


ki 


Very Good 


Moderate 


Second 


k\ 


Very Poor 




First 


k 2 


Good 


Huge 


Second 


kz 


Acceptable 



Although the theory presented suggests that in this case the feedback 
optimal control is the only variable for which sensitivity improvement 
occurs, we see here that the sensitivity improvment in the control is evidently 
linked with sensitivity improvement in the response of x,. This example also 
illustrates the notion that large Q lead to less sensitive closed-loop systems 
than smaller Q . Although the entries of the F matrix undergo substantial 
change, the response of to a typical initial condition varies but little, when 
the design based on Q z is used. 

The first sensitivity result for linear optimal systems was achieved in 
[6], where it was noted that the quantity — k'x occurring in the single-input 
problem could be associated with a sensitivity reduction result. The corre- 
sponding result for multiple-input systems is derived in [7]. Versions of these 
results for time-varying optimal regulators are described in [8], 

The latter material of this section, dealing with the state sensitivity 
question, appears in [5], where the necessity for constraining the allowed 
sorts of parameter variations is not directly indicated, and also in [9]. In 
both references, a restriction is made on the F matrix and g vector of the sys- 
tem, which we have avoided here. 

An entirely different sensitivity question arises when one asks what is 
the sensitivity of the optimal performance index of an optimal system — in 
particular, the linear regulator — to parameter variation, and how do the 
sensitivities for open- and closed-loop implementations of the optimal control 
vary. This question is discussed in [9]. The somewhat surprising result for the 
linear regulator is that the two sensitivities are the same. Various other sen- 
sitivity problems are discussed in reference [10]. 

Problem 7.2-1 . Suppose an optimal control law u = k^x is found for a single- 
input system with F and g given by 
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Show that the sensitivity of the states of the system is less for closed- than for open- 
loop control implementations, provided that only the two following types of param- 
eter variation are permitted: 

1. One or more of the a t are varied slightly. 

2. The nonzero entry of g is varied slightly. 

Problem 7.2-2. Attempt to generalize the development of Eqs. (7.2-7) through 
(7.2-13) to multiple-input systems, and show where the generalization falls down. 

Problem 7.2-3. Suppose two optimal regulator designs are carried out with 
the same F, G, Q , and R matrices and with two values of a. Call them 0C), a 2 , with 
the constraint OCi > a 2 > 0. Show that the a x design offers greater sensitivity im- 
provement than the oc 2 design. [Hint: Recall that P ai — P a2 is positive definite at 
a critical point in your calculations.] 



7.3 THE INVERSE REGULATOR PROBLEM 



In this section, we consider a problem of somewhat theoretical interest, 
the solution of which nevertheless sheds light on the preceding sensitivity 
ideas. In the next section we shall apply the solution of the problem in a 
further discussion of the “pole-positioning problem,” i.e., the task of choos- 
ing an optimal feedback law that also achieves desired pole positions. 

We ask the following question. Given a linear system 

x = Fx + Gu (7.3-1) 

with control law 

u - K' jc, (7.3-2) 

is there a positive definite R and nonnegative definite Q such that (7.3-2) 
is the optimal control law for the system (7.3-1) with performance index 

/• 00 

V(x(t 0 ),u(-),t 0 ) = {u’Ru + x’Qx)drt (7.3-3) 

J t 0 

Obviously, for arbitrary F, G, and K we could not expect (7.3-2) to be 
optimal for some Q and R. Accordingly, we are led to ask whether certain 
constraints can be imposed on F, G, and K, which will ensure that (7.3-2) is 
an optimal control; furthermore, we might be interested in computing the 
Q and R matrices appearing in (7.3-3). 

We shall give a detailed consideration of this problem only for the scalar 
input case, because the manipulations required in the vector input case are 
particularly intricate. We shall, however, subsequently state a result for the 
vector input case. 

More or less, the only regulator systems of interest are those that are 




136 SENSITIVITY ASPECTS OF OPTIMAL REGULATORS 



Ch. 7 



completely controllable and asymptotically stable. Accordingly, we shall 
suppose the following. 

Assumption 1. 3d. The pair [F,g] is completely controllable. 

Assumption 7.3-2. The eigenvalues of F -f gk' all have negative real parts. 

[Recall that the closed-loop system is x = (F + gk')x.] 

It turns out that to keep the calculations simple, it is convenient to 
impose one further assumption temporarily. 

Assumption 7.3-3. The pair [F, k] is completely observable. (Thus, the feedback 

control function can only be identically zero if the system state is identically 

zero.) 

Problem 7.3-1 asks for a demonstration that this constraint may be 
removed. 

We now claim that the scheme (7.3-1), (7.3-2), with scalar input, is 
optimal [i.e., there exist a Q and an R with appropriate constraints such that 
(7.3-2) is the optimal control for the performance index (7.3-3)], provided 
that, 

|1 -~kXjcoI-FY'g\> 1 (7.3-4) 

for all real co. Equation (7.3-4) is precisely the sensitivity improvement condi- 
tion for the closed-loop system defined by (7.3-1) and (7.3-2). Moreover, if 
(7.3-1) and (7.3-2) are optimal, then (7.3-4) is known to hold, by the material 
of Sec. 7.2. Consequently, with the assumptions of complete controllability 
and asymptotic stability just given, (7.3-4) is both necessary and sufficient. 
This result constitutes a connection between the modern approach to linear 
control system design and an approach of classical control of very long 
standing, to which we referred at the start of the chapter. Although the full 
implications of (7,3-4) in respect to sensitivity reduction have not been known 
until recently, a sufficient number of them have been known for many years 
for designers to have attempted to ensure the satisfaction of (7.3-4) or some 
suitable variant using classical procedures. 

Equation (7.3-4) is a condition on the transfer function k\jcoI — F)~ l g . 
Accordingly, it is invariant with respect to changes of the coordinate basis 
used for defining the state vector in (7.3-1) and (7.3-2) — i.e., if Fis replaced 
by F t = TFT 1 , g by g x = Tg, and k! by k\ = k'T~\ we have that k\(jcoI 
— F l )~ 1 g 1 = k'ijcol — F)' l g. Therefore, there is no loss of generality in 
assuming a special coordinate basis; we shall avail ourselves of this oppor- 
tunity and require (as we may by the complete controllability constraint, 
see Appendix B) that the coordinate basis be chosen to yield 
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We assume that 

k = [k , k 2 • • • k n ]. 

Make the definition 

det(s/ — F) = ^(s). 

Then the following three relations hold: 

y/(s) = j" + + * • • + a l9 

1 “ 



(si- F)~'g = 



y/(s) 



and 




k'(sl — F)~'g 



.n- 1 



+ k m . i s?- 2 + ••• +*, 



We shall now use the relation (7.3-4) to construct a Q matrix; 
take the R matrix in the performance index (7.3-3) to be unity. 
With the further definition 



p(s) = s* + (a„ - k n )s n ~ l H + (a x - k x ), 

it follows from (7.3-4) that 

[1 - k'i-jml - F)-'*][l - k'UcoI - F)-g]= 

W(~J co)y/(jco) 



or 

p(~jm)p(jm) - y/( ~joj)y/( jco) 

y/(-jco)i//(ja>) — 

Consequently, the numerator polynomial of this expression can be 
as 



(7.3-5) 

(7.3-6) 

(7.3-7) 

(7.3-8) 



(7.3-9) 
we shall 

(7.3-10) 

> 1, 

(7.3-11) 

(7.3-12) 

factored 



p(-ja>)p(jco) - ip(~jco)ij/(j(o) = m(-j(o)m(jco ) (7.3-13) 
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for some polynomial 

m(s) = d n s n ~ 1 + d^^s"- 2 + • • • -f d x (7.3-14) 



with all zeros possessing negative real parts. [Note that m(s ) has degree at 
most n — 1, as inspection of the coefficients of the s n terms of y/(s) and p(s ) 
will show.] 

Equations (7.3-11) through (7.3-13) combine to yield 
[1 - k'i-jcol - F)-'g][\ - k'fjtol- FY'g] 

= i , m(-jco)m(ja>) 

= 1 + g'(-jcol - F)~ l dd'{jo)I - FY’g (7.3-15) 
where the vector d is given by 

d' = [d t d 2 d n ] (7.3-16) 

and we have used the relation, analogous to (7.3-9), that 



d’(jcoI~Fr 1 g = 





mjjoi) 

«) 



Now, if k were the optimal control law corresponding to R~ I and a 
certain matrix Q in (7.3-3), the results of Sec. 5.2 [see the fundamental 
equation (5.2-9)], would require that 



[1 _ k'i-jwI-FY'g ] [1 - k'UcoI- FY'g ] 

= 1 + g’i-jcol - F'Y'Q(jcoI - FY'g. (7.3-17) 

The similarity between (7.3-15) and (7.3-17) then suggests that with 
Q = dd' in (7.3-3), the control law k might be optimal. Note that mere satisfac- 
tion of (7.3-1 5) by some k will not necessarily guarantee the optimality of k; Eq. 
(7.3-15) is a necessary, but not a sufficient, condition on k. Nevertheless, we 
can now prove that k is, in fact, optimal for the choice Q = dd’, R = 1. 

Suppose k is not optimal, and that the optimal control is, instead, 
u = k'x. Using the fundamental result of Chapter 5, Eq. (5.2-9), it follows 
that 

[1 - K\-jcoI-F)-'g}[ 1 - k'(jcoI - FY'g] 

= 1 + g'i-jcol - F'Y'dd’(jcoI - FY'g. (7.3-18) 

Comparison of (7.3-15) and (7.3-18) gives immediately that 

[1 - k\-jmI-FY'g][\ - k'UcoI -FY'g] 

— [1 ~ k’(—jcoI — F)-‘g][l — k'ijcol — FT'g], (7.3-19) 

We shall now show that this equation implies that k — k. 

Define a polynomial p(s ) in the same way as p(s) [see (7.3-10)] except that 
k t in the definition is replaced by k, for each /. Thus, (7.3-18) and (7.3-19) 
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imply that 

p(-jco)p(jco) = p(-jco)pUco) = ! , m(-j(D)m(jCD) (7.3-20) 

y/(—jco)y/(j(o) y'(—ja>)y/(jcD) Wi-j^vO 0 *) 

Since the coefficients of p(s) and p(s) are the entries of the vectors k and k, 
the proof will be complete if we can show that p(s ) — p(s). Now it is easy to 
check that 

p(s) =-- det (si - F - gk f ) (7.3-21) 

and 

p(s) — det (si — F — gk'). (7.3-22) 

[This follows from the definition of the polynomials, together with a minor 
modification of the calculations yielding (7.3-8) from (7.3-5) and (7.3-7).] 
Because F + gk f is known to have all eigenvalues with negative real parts, 

p(s) is uniquely determined as the spectral factor of the polynomial 

p(—j(o)p(jco), which has all zeros in the left half-plane. 

If we can show that p(s) has the property that it, too, has all zeros in the 
left half-plane, it will follow from (7.3-20) that p(s) = p(s). From (7.3-22), 
it is clear that p(s) will have all its zeros in the left half-plane if the optimal 
closed-loop system, x = (F + gk f )x , is asymptotically stable. A sufficient 
condition for this is that the pair [F, d ] be completely observable. 

Consequently, the desired result follows if the pair [F, d ] is proved to be 
completely observable. We now show by contradiction that this must be the 
case. Thus, suppose [F, d] is not completely observable. Then the transfer 
function d'(sl — F) _1 g, when expressed as a ratio of two polynomials with no 
common factors, must (see Appendix B) have denominator polynomial of 
degree less than the dimension of F. Since d'(sl — F)~ l g = m(s)/y/(s), and 
the degree of y/ is equal to the dimension of F, it follows that m and y/ have 
a common factor. Because m(s) has all zeros with negative real part, the com- 
mon factor must possess the same property. Equation (7.3-20) implies that 
p(—s)p(s) and y/(s) must have the same common factor as m(s) and y/(s ), 
and since p(s) has all its zeros with negative real part, p(s) and y/(s) must have 
this same common factor. Therefore, y/(s) — p(s) and y/(s) have a common 
factor, and the transfer function 

k'(sl - F)~ l g = (7.3-23) 

is expressible as a ratio of two polynomials, with the denominator polynomial 
of lower degree than the dimension of F. This (see Appendix B again) 
contradicts the complete controllability of [/% g] and complete observability 
°f 17% *]• Therefore, our original premise that [F, d ] is not completely observ- 
able is untenable, and the proof is complete. 

Let us now summarize the preceding material. We posed the problem 
of deciding when the feedback system of (7.3-1) and (7.3-2) resulted from an 
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optimal regulator design. Then, with several restrictions — viz., complete 
controllability, asymptotic stability, and a further restriction of complete 
observability, which Problem 7.3-1 removes— we claimed that the frequency 
domain condition requiring the return difference to be never less than unity 
was necessary and sufficient for optimality. Using the frequency domain 
condition, we constructed a performance index, which, we claimed, led to 
an optimal control the same as the one prescribed. We were then able to 
verify this claim. This result was first established in [6], 

There exists a corresponding result for multiple-input systems that 
we can now state. We suppose that we are given the scheme of (7.3-1) and 
(7.3-2), with complete controllability of [F, G] and asymptotic stability of the 
closed-loop system. We suppose too that [F, K] is completely observable, 
although this restriction is presumably removable. Then, if for some positive 
definite symmetric R we have 

[/- R ,/2 K' 0 {jcol - F)-'GR- U2 }* V ~ R in K' 0 {jcoI - F)-'GF->' 2 ] > /, 

(7.3-24) 

it can be shown that there is a nor.negative definite symmetric Q such that 
the prescribed control law (7.3-2) is optimal for the performance index 
(7.3-3), [7]. 

We remark again that, other than in the application given in the next 
section, the utility of this section lies in the fact that new illumination of the 
sensitivity reduction result is provided. In broad terms, feedback systems 
where sensitivity is reduced are the same as optimally designed systems. 



Problem 7.3*1 . Consider the system 



with control law 




0 ” 
^ 22 - 



* + 




u 



Lgz J 



u ~ [k\ 0]x 



where [Fj t , g t ] is completely controllable, [F u , k i ] is completely observable, and the 
closed-loop system is asymptotically stable. Suppose, moreover, that the magnitude 
of the return difference is no smaller than unity at all frequencies. Show that there 
is a matrix Q such that the control law is optimal for the performance index 
j7 0 (« 2 + x'Qx)dt[Hint: Start by considering the feedback system x { =F n x i +g l u, 
u = k[xi.] 



Problem 7.3-2. In the course of establishing the result that 

|1 — k'(jcoI ~ F)~ l g\>\ 

for all real 0) implies optimality of the control law u = k'x for the system x — Fx + gu, 
we constructed a particular performance index for which u = k'x was the minimizing 
control. In this performance index, the Q matrix was of the form dd' for a vector 
d. Show the following: 
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1 . In general, the choice of d is not unique, but only a finite number of choices 
are possible. 

2. If Q is not restricted to being of rank 1, there are an infinite number of Q 
matrices for which u = k'x is the associated optimal control. 

Problem 7.3-3. Suppose we are given a system x = Fx + gu with feedback 
law u = k'x. Suppose also that the pair [ F , #] is completely controllable, that the 
pair [F, k\ is completely observable, and that the closed-loop system x = (F + gk')x 
has degree of stability greater than a. Show that u = k'x is an optimal control law 
for a performance index of the form |^e 2otf (« 2 + x'dd'x)dt, provided that 

|1 — k'[(jco — a)/ — F]~ l g\ > 1 

for all real 



7.4 FURTHER DISCUSSION OF THE POLE-POSITIONING 
PROBLEM 

In this section, we return to the discussion of the pole-positioning prob- 
lem for single-input systems. We suppose that we are given the system 

x = Fx + gu (7-4-1) 

and we desire a control law 

u = k'x (7-4-2) 

which is optimal for the usual sort of quadratic loss function, and such that 
the eigenvalues of F + gk' take on desired values, at least approximately. 
The approach we adopt here is based on application of the inequality 

|1 ~k'(jcoI- F)~'g\> 1 (7.4-3) 

for all real c o, which we know is necessary and sufficient for u ~ k'x to be 
optimal, given that (1) the pair [F, g] is completely controllable, and (2) 
the closed-loop system x = (F + gk')x is asymptotically stable. 

We shall also relate our discussion here to that given earlier in Chapter 
5, Sec. 5.4, where we discussed the achieving of desired poles by taking very 
large p in the performance index [u 1 + p{d'x) 2 ) dt , and appropriately 
choosing d. 

First, we find an equivalent statement to (7.4-3). Observe that 
det [si ~{F + gk ')] = det {[si - F][I - (si - F) 'gk']} = det [si - F] X 
det [I - (si - Fy'gk'] = det [si - F][l - k'(sl - F)~'g}. [Here, we have 
used the relations, see Appendix A, det (AB) = det A det B and det (/„ + AB) 
= det (/„ + BA), where the matrices have appropriate dimensions.] Con- 
sequently, (7.4-3) and the definitions 

yj(s) = det (si - F) and p(s) = det (si - F - gk') 



imply that 
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p(-j(0)p( j<X>) , 

V(-j(o)vU(o) 

or that 

p{~~jco)p{jco) > ^(— yo>)v/(yco) (7.4-4) 

for all real co. Notice also that Eq. (7.4-4) implies (7.4-3). 

Now suppose that a list of n desired closed-loop pole positions is given 
for (7.4-1). These pole positions determine uniquely the characteristic poly- 
nomial p(s) = s n + (a n ~ k n )s n ~ { + • • • + (a, - *,), and, since (7.4-4) 
implies (7.4-3), these closed-loop pole positions are achievable with an optimal 
control law u — k'x if and only if (7.4-4) is satisfied. In other words, (7.4-4) 
provides a test that will decide whether a set of n prescribed closed-loop poles 
are optimal or not. Moreover, if they are optimal, it is possible to determine 
an associated control law and performance index using the ideas of the previ- 
ous section. The pole-positioning problem is thereby solved. Note that in 
the earlier treatment, the positions of at most ( n — 1) poles were prescribed 
(with the remainder lying well in the left half-plane). By contrast, n pole 
positions are prescribed here. 

What, though, if the list of n poles is not such that (7.4-4) holds? It is 
obviously not possible to choose an optimal control law generating these 
desired poles, and some concession must be made. One such concession is 
to relax the positioning constraint on one real pole (say), and permit its posi- 
tion to be adjustable. Then, of course, we are returning to a situation similar 
to that discussed earlier, in Sec. 5.4. But the use of (7.4-4) can give us addi- 
tional insight into the problem. Specifying ( n — 1) poles and leaving one 
unspecified on the real axis means specifying that 

P(jco) = (jco + <x)p x (jco) (7.4-5) 

where a is adjustable, whereas p x (s) has zeros which are the (n — 1) specified 
poles. Then (7.4-4) may be written 

{co 1 + a ^p.i-jcojp.ijco) > y/{"jco)y/{jco). (7.4-6) 

The coefficients of co 2n on both sides are the same, and it is therefore possible 
to find a suitably large value of a such that the inequality holds, irrespective 
of the polynomials y/(s) and />,(s), although the actual value of a depends, of 
course, on these polynomials. For this and larger values of a (corresponding 
to a closed-loop pole at or to the left of a certain point on the negative real 
axis), the closed-loop system defined via (7.4-5) is optimal. 

In summary, if a set of n desired closed-loop poles fails to satisfy the 
inequality criterion, adjustment of one pole (by moving it in the negative 
direction along the negative real axis sufficiently far) will result in an optimal 
set of poles, i.e., a set such that the inequality (7.4-4) is satisfied. 

We recall that the method of Sec. 5.4, also yielded a set of closed-loop 
poles among which one or more were a substantial distance from the origin. 
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With both schemes, a certain difficulty may arise. With one or more 
poles well in the left half-plane, one or more of the entries of the feedback 
gain vector must be large. [This follows, for example, by noting that the sum 
of the closed-loop poles is — {a n — k n ), the product is (— l)”(a t — k x ), etc. 
Consequently, the magnitude of the control applied to system (7.4-1) may be 
large — large enough, in fact, to cause saturation. This conclusion is also borne 
out by the fact that making p large in the performance index [u 1 + 
p(d r x ) 2 ] dt serves to penalize nonzero states rather than nonzero control. 
Therefore, any pole-positioning scheme should only be used with caution. 

Finally, given p(s ) and y/(s ), we note that the actual computations 
required to check whether the inequality (7.4-4) holds are straightforward. 
Of course, the problem of checking (7.4-4) is equivalent to one of checking 
whether a polynomial r(*) in c o 2 is nonnegative for all co. Procedures based 
on Sturm’s theorem or the equivalent Routh test (see [11] and [12], respectively) 
provide a check in a finite number of steps involving simple manipulations 
of the coefficients of r(-). For a survey of other tests, see [13]. 

Problem 7.4-1 . Consider the system 




Show that it is not possible to choose an optimal control law such that the closed- 
loop poles are both at s = — Adjust the closed-loop poles until an optimal design 
is achievable. 

Problem 7.4-2. For the same system as Problem 7.4-1, show that it is never 
possible to obtain complex closed-loop poles which are also optimal for some 
quadratic performance index. 

Problem 7.4-3. Consider the system 

"0 1“| TO' 

X = X + u. 

L0 Oj LlJ 

Show that closed-loop poles which are optimal for a quadratic performance index 
must have a damping ratio of at least 1 /\/~2. 

Problem 7.4-4. Suppose that the control law u = k'x is optimal for the com- 
pletely controllable system x — Fx + gu 9 and the closed-loop system is asympto- 
tically stable. Show that 

I nu,(F)] i < | mMF + gk')} |. 
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STATE ESTIMATOR DESIGN 



8.1 THE NATURE OF THE STATE ESTIMATION PROBLEM 

The implementation of the optimal control laws discussed hitherto 
depends on the states of the controlled plant being available for measurement. 
Frequently, in practical situations, this will not be the case, and some artifice 
to get around this problem is required. Before outlining the various 
approaches that may be used, let us dismiss upon practical grounds one 
theoretically attractive state estimation procedure. 

Given a completely observable system, the state vector may be con- 
structed from linear combinations of the output, input, and derivative of 
these quantities, as we now show. Consider for simplicity a single-output 
system: thus, 



x = Fx + Gu (8.1-1) 

y = h'x. (8.1-2) 

Differentiation of (8.1-2) and substitution for x from (8.1-1) leads to 

y h'Gu = h'Fx. (8.1-3) 

Differentiating again, and substituting again for x, we get 

y — h'gu — h'Fgu = h'F 1 x. (8.1-4) 

Continuing in this way, we can obtain a set of equations which may be summed 
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up as 



h'F 



x 




or 

z' = x?[h F'h (F'y-'h] (8.1-5) 

with z a row vector, the entries of which consist of linear combinations of 
y, u , and their derivatives. 

Now one of the results concerning complete observability is that the 
pair [F, H] is completely observable if and only if the matrix [H F'H • • • 
(F')"' 1 //] has rank n. (A demonstration that this result follows from one of 
the earlier definitions given for complete observability is requested in Prob- 
lem 8.1-1. See also Appendix B.) In the scalar output case, the matrix 
[h F'h • • • (F') n ~ { h\ becomes square, and thus has rank n if and only if it is 
nonsingular. 

Therefore, with the system of (8.1-1) and (8.1-2) completely observable, 
Eq. (8.1-5) implies that the entries of the state vector jc are expressible as 
linear combinations of the system output, input, and their derivatives. 

From the strictly theoretical point of view, this idea for state estimation 
works. From the practical point of view, it will not. The reason, of course, 
is that the presence of noise in u and y will lead to vast errors in the com- 
putation of x, because differentiation of u and y (and, therefore, of the asso- 
ciated noise) is involved. This approach must thus be abandoned on practical 
grounds. 

However, it may be possible to use a slight modification of the preced- 
ing idea, because it is possible to build “approximate” differentiators that 
may magnify noise but not in an unbounded fashion. The rate gyro, normally 
viewed as having a transfer function of the form Ks , in reality has a transfer 
function of the form Kas/(s + a ), where a is a large constant. Problem 8.1-2 
asks the student to discuss how this transfer function might approximately 
differentiate. 

Generally speaking, a less ad hoc approach must be adopted to state 

estimation. Let us start by stating two desirable properties of a state esti- 
mator : 



1. It should be a system of the form of Fig. 8.1-1, with inputs consisting 
of the system input and output, and output x e (t) at time t , an on- 
line estimate of x(t). This should allow implementation of the 
(optimal) control law u = K'x e , according to the scheme of Fig. 8.1-2. 
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Fig. 8.1 -2 Use of estimator in implementing a control law. 



2. It should function in the presence of noise. Preferably, it should be 
possible to optimize the action of the estimator in a noisy environ- 
ment — i.e., to ensure that the noise has the least possible effect when 
the estimator is used in connection with controlling a system. 

As we shall show, these properties are both more or less achievable. Esti- 
mators designed to behave optimally in the presence of noise turn out to con- 
sist of linear, finite-dimensional systems, if the system whose states are being 
estimated is linear and finite-dimensional. Moreover, if the dimensions of the 
system whose states are being estimated and the estimator are the same, and 
if the system whose states are being estimated is time invariant and the 
associated noise is stationary, the estimator is time invariant. Also, all these 
properties hold irrespective of the number of inputs and outputs of the sys- 
tem whose states are being estimated. 

A further interesting property of such estimators is that their design is 
independent of the associated optimal feedback law K' shown in Fig. 8.1-2, 
or of any performance index used to determine K\ Likewise, the determina- 
tion of K' is independent of the presence of noise, and certainly independent 
of the particular noise statistics. Yet, use of the control law u = K'x e turns 
out to be not merely approximately optimal, but exactly optimal for a modi- 
fied form of performance index. This point is discussed at length in Chapter 
9, Sec. 9.4. 

If satisfactory rather than optimal performance in the presence of noise 
is acceptable, two simplifications can be made. First, the actual computa- 
tional procedure for designing an estimator (at least for scalar output systems) 
becomes far less complex. Second, it is possible to simplify the structure of the 
estimator, if desired. In other words, the designer can stick with the same 
estimator structure as is used for optimal estimators (although optimality 
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is now lost), or he may opt for a simpler structure. In general, the estimator 
with simpler structure is more noise prone. The nature of the structural 
simplification is that the dimension of the estimator is lowered, by one for a 
single-output system and sometimes by a larger number for a multiple-output 
system. 

Because they are simpler to understand, we shall discuss first those esti- 
mators that are not designed on the basis of optimizing their behavior in the 
presence of noise; this class of estimators may be divided into two subclasses, 
consisting of those estimators with the same dimension as the system and 
those with lower dimension. Then the class of estimators offering optimal 
noise performance will be discussed. The next chapter will consider the use 
of estimators in implementing control laws, and the various properties of the 
associated plant controller arrangement. 

One point to keep in mind is that the output of the estimator at time 
t j x e (t), is normally an estimate rather than an exact replica of the system 
state x(t), even when there is no noise present. In other words, no estimator 
is normally perfectly accurate, and thus feedback laws using an estimate 
rather than the true states will only approximate the ideal situation. Despite 
this, little or no practical difficulty arises in controller design as a result of the 
approximation. 

Problem 8.1-1. We have earlier adopted the following definition: [F y H] is 
completely observable if and only if H'e Ft x 0 = 0 for all t implies * 0 = 0. Show 
that a necessary and sufficient condition for complete observability according to 
this definition is that rank [H F'H •• • (Fy~'H) = 0. {Hint: Suppose that 
H'e Ft x Q = 0 for all t and some nonzero * 0 . Show by differentiating and choosing 
a specific t that Xq[H F'H • • * ( F') n ~ l H ] = 0. For the converse, suppose that 
x'q[H F'H • • • (F') n ~ { H] = 0 for some nonzero x 0i and use the fact that 
e Ft = 0C Q (t)I + + oc for certain scalar functions a, (I), to 

conclude that H'e Ft x 0 — 0 for all t .] 



Problem 8.1-2. In what sense can the transfer function Ks be approximated 
by the transfer function Kas/(s + a)? What is the significance of the size of al 
Discuss the effect of the noise with the two transfer functions (e.g., evaluate the 
output spectra for one or two types of input noise spectra). 



Problem 8.1-3. Show that the circuit of Fig. 8.1-3 can be used as an approxi- 
mate differentiator. Discuss the effect of noise in e { . 



° 




Fig. 8.1-3 Circuit for Problem 8.1-3. 
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8.2 NON STATISTICAL ESTIMATOR DESIGN WITH FULL 
ESTIMATOR DIMENSION 



In this section, we shall consider the design of estimators for systems of 
the form 



x = Fx +Gu (8.2-1) 

y = H'x . (8.2-2) 

We shall assume that F, G , and H are time invariant, although it is possible 

to extend the theory to time-varying systems. The estimators will be of the 
general form 

= F e x e + G u u + G u y (8.2-3) 

with F e of the same size as F. Equation (8.2-3) reflects the fact that the inputs 
to the estimator are the input and output associated with Eqs. (8.2-1) and 
(8.2-2), hereafter called the plant input and output. The estimator itself is a 
linear, finite-dimensional, time-invariant system, the output of which is the 
estimated state vector of the system. 

Before we give a detailed statement specifying how to choose F ei G le , 
and (7 2e , it is worthwhile to make two helpful observations: 

1. It would be futile to think of constructing an estimator using (8.2-3) if 
the plant equations (8.2-1) and (8.2-2) did not define a completely 
observable pair [F, H), because lack of complete observability implies 
the impossibility of determining, by any technique at all, the state of 
the plant from the plant input and output. 

2. One might expect the estimator to be a model for the plant, for sup- 
pose that at some time t 0 , the plant state x(t 0 ) and estimator state 
x e (t Q ) were the same. Then the way to ensure that at some subsequent 
time x e (t) will be the same as x(t) is to require that the estimator, in 
fact, model the plant — i.e., that 

x e - Fx e + Gu. (8.2-4) 

Clearly, though, Eq. (8.2-4) is not satisfactory if x e (t 0 ) and x(t Q ) are differ- 
ent. What is required is some modification of (8.2-4) that reduces to (8.2-4) 
if *«(*<>) an< 3 *Oo) are the same, and otherwise tends to shrink the difference 
between x e {t) and x(t) until they are effectively the same. Now we may ask 
what measure we could physically construct of the difference between x e (t) 
and x(t). Certainly there is no direct measure, but we do have available H'x , 
and therefore we could physically construct H f [x e (t) ~ x(t)]. Hopefully, the 
complete observability of the plant would ensure that, over a nonzero interval 
of time, this quantity contained as much information as x e (t) — x(t). 
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These considerations suggest that instead of (8.2-4), we might aim for 

= F*. + Gu + K e H'[x e - x] (8.2-5) 

as an estimator equation. This scheme is shown in Fig. 8.2-1. The equation 
has the property that if x and x, are the same at some time then they 
will be the same for all f > , 0 , the third term on the right-hand side being 
zero for all t. Furthermore, it has the property that possible judicious selec- 

r^L ~'- e ;’ Jl,d,C,OUS introduction of a signal into the estimator reflecting 
the difference between and y(t) = H'x{t)-mzy ensure that the error 

between x£t) and x(t) becomes smaller as time advances. Let us now check 
this latter possibility. 




dt (X ~ * «) = F ( x - *.) + K.H'(x - x e ) 

- (F i K e H')(x - Xt ) (8 2 _ 6) 

follow, Consequently if the eigenvalue, of (F + K. H ) have negative real 

effectively trlekTwter ? .“‘''lY** 0 "'""*' "“*• and 

F + K.lr. ,n,erval dcterm ™ d ^ the eigenvalues of 

of "“(8 2 h 2) a ro! a ,h' i ' y °? einP “‘ a " d 

of the plan,. B, rough aUn,,™ 2SS 
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bility of an estimator design of the form of Eq. (8.2-5), which may be rewritten 
as 

x e =-~(F + K e H')x e + Gu - K,y. (8.2-7) 

[Figure 8.2-2 shows how this equation can be implemented.] Then we were 
able to conclude that if K e could be chosen so that the eigenvalues of 
F + K e H f had negative real parts, Equation (8.2-7) did, in fact, specify an 
estimator, in the sense that x e ~ x approaches zero at some exponential rate. 
Note that at this stage, we have not considered any questions relating to the 
introduction of noise. 




Fig. 8.2-2 Estimator with minor simplification. 



The natural question now arises: When can K e be found so that F + 
K e H’ has arbitrary eigenvalues? The answer is — precisely when the pair 
[F, H] is completely observable ! (The earliest statement of this result appears 
to be in [1], but it or an equivalent result has since received wide exposure — 
e.g., [2] through [4].) Therefore, aside from the computational details involved 
in determining K e , and aside from checking the noise performance, we have 
indicated one solution to the estimator problem. 

The scheme of Fig. 8.2-1, earlier regarded as tentative, has now been 
shown to constitute a valid estimator. The estimator is a model of the plant, 
with the addition of a driving term reflecting the error between the plant out- 
put y = H'x and the variable H'x e , which has the effect of causing jc* to 
approach x. Figure 8.2-2 shows an alternative valid estimator representation 
equivalent to that shown in Fig. 8.2-1. In this second figure, however, the 
concept of the estimator as a model of the plant becomes somewhat sub- 
merged. 

Let us now consider the question of the effect of noise on estimator 
operation. If noise is associated with u and y, then inevitably it will be 
smoothed by the estimator; and if white noise is associated with either u or 
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y (i.e., noise with a uniform power spectrum), the spectrum of the noise in 
x e will fall away at high frequencies. In general, the amount of output noise 
will depend on the choice of K e , but the problem associated with passing noise 
into a differentiator will never be encountered. The choice of K e also affects 
the rate at which x e approaches x, because this rate is, in turn, governed by 
the eigenvalues of F +■ K e H\ It might be thought that the best way to choose 
K e would be to ensure that the eigenvalues of F + K e H' had real parts as 
negative as possible, so that the approach of to x would be as rapid as 
possible. This is so, with one proviso. As the eigenvalues of F + K e H f get 
further into the left half-plane, the effective bandwidth of the estimator could 
be expected to increase, and, accordingly, the noise in x e due to noise in u 
and y could be expected to increase. Therefore, noise will set an upper limit 
on the speed with which x e might approach x. The situation with using differ- 
entiation to estimate x is that the noise becomes infinite, with the estimation 
time infinitesimally small. The use of estimators of the form just described 
is aimed at trading off speed of estimation against minimization of loss of 
performance due to noise. The optimal estimators of Sec. 8.4 essentially 
achieve the best compromise. 

In the remainder of this section, we shall discuss the problem of comput- 
ing K e . In the next chapter, we shall return to the investigation of the perfor- 
mance of the estimator in an optimal control system, and to minor variants 
of it. 

We shall first discuss the question of computing K e for a single-output 
system. In this case, K e becomes a vector, and will accordingly be denoted by 
k e . The problem of finding k e to produce arbitrary eigenvalues for the matrix 
F + k e h’ is easy if F and h have the following form: 
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( 8 . 2 - 8 ) 



F + k e W = 



(8.2-9) 
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and 

det [si — (F + k e W)] = s n + {a n - k ne )s n + • • • +{a { - k ie \ 

( 8 . 2 - 10 ) 

as may easily be shown. Now specification of the eigenvalues of F + k e ti 
is equivalent to specification of the characteristic polynomial of F + k e h\ 
or, equivalently, the coefficients a, - k ie in (8.2-10). Therefore, with the a t 
known and the coefficients a L ~ k ie specified, the vector k e is immediately 
determined. 

When F and h are not in the form just shown, it is necessary to trans- 
form them into this form via an appropriate basis change, to choose a k e 
in this new basis, and then to determine the k e in the old basis. In precise 
terms, we define the matrix 




( 8 . 2 - 11 ) 



where s n + a,^ 1 + • * • + a x is the characteristic polynomial of F. 

Recall that the matrix [h F'h • - • (F') n ~ l h] has rank n if and only if 

[F,h] is completely observable. Therefore, the matrix T of (8.2-11) is non- 
singular if and only if [F, h ] is completely observable. 

The significance of T is that, as we shall show. 




( 8 . 2 - 12 ) 

Accordingly, if k e is determined so that F + k e h' has desired eigenvalues 
(as is easy to do), it follows that with 



K - T-'k e (8.2-13) 

the matrix F + kji has the same eigenvalues. 

To verify the claim concerning the structure of F and h , we follow an 
idea of Tuel [5] applied originally to completely controllable systems. Define 
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the row vectors t ' 2 , ...,/' sequentially through 

t' n = h' 

t'i = t' i+x F + a l+l h' i < n. (8.2-14) 

Observe that 

t\F= t' 2 F 2 + aJi'F 

= t'iF 3 + a s h'F 2 + a 2 h'F 



= t'nF" + a^KF- 1 + • • • + a z h'F 
= h'[F n +a„.,F n -‘ + • • • + a 2 F ] 



f',F = -a,A' (8.2-15) 

on using the fact that F satisfies its characteristic equation. Now, 
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where we have made use of Eqs. (8.2-14) and (8.2-15). Also, 



Therefore, with 



[0 0 • 






= 4 = h'. 





(8.2-16) 



Eqs. (8.2-12) follow, with F and h possessing the desired structure. Using the 
definitions (8.2-14), it is also straightforward to verify that T has the form of 
Eq. (8.2-11). 
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Let us now consider a simple example. Suppose we are given the plant 
equations 






(8.2-17) 



y = [2 -i]* 



which describe a second-order system with transfer function ( s + 1 )/s(s — 1). 
We shall provide an estimator design such that the eigenvalues of F + k e h' 
are both at —10. Then x ~ x e will decay as fast as e ' 10t . First, the matrix 
T must be found which will take F and h into the special format of Eq. 
(8.2-12). We have det (si — F) = s 2 — s , and therefore, from Eq. (8.2-11), 



Also, 



Then 



and therefore, 
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so that 

0 -100" 
1 - 20 _ 

has two eigenvalues of —10. Then, 



F + k e h' = 




T~ x k, 



60.5“ 

100 



Using (8.2-7), we find the estimator equation to be 
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One interesting conclusion from this equation is that apparently noise asso- 
ciated with the output measurements y will have a significantly greater effect 
in terms of introducing noise into x* than will noise in the input measurement 

u. 
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We now turn to a discussion of estimator design for multiple-output 
plants. The computational procedure leading to selection of a matrix K e 
such that F + K e H' has arbitrary eigenvalues for a multiple output plant is 
exceedingly involved, and will not be studied in generality here. Discussion 
may be found in [1] through [4] and [6] through [8]. 

However, there are two classes of multiple output systems where the 
preceding theory is sufficient. The first class consists of those multiple-output 
systems for which the state vector is observable from one output alone — 
i.e., if h i is the zth column of the H matrix, [F, h] is completely observable for 
one L Observer design then can proceed using the scalar output variable 
h'x and disregarding the other components of the output. One would expect 
this to be far from optimal in a noisy situation, since information is being 
thrown away in estimating the state variable. 

The second class of plants are those made up of a cascade of small plants. 
Frequently in practice, various elements of an overall system may be indivi- 
dually distinguishable, and the system will have the general form of Fig. 
8.2-3, where for convenience we have indicated the individual small plants by 
transfer functions w/s) rather than by state-space equations. The overall 
plant transfer function is h^) = JJ w.(s). 




Fig. 8.2-3 Cascade of small plants providing a large plant. 



One way to carry out state estimation in this case is to use a number 
of small estimators, call them E u E 29 . . . , E n with estimator E t estimating 
the states of the small plant with transfer function wfe). Estimator F, is, 
of course, driven by the input and output of w { (s) 9 and will have the same 
dimension (when viewed as a linear system) as does w t . Figure 8.2-4 shows the 
composite estimation scheme. Evidently, the dimension of the overall esti- 
mator is the sum of the dimensions of the individual w t (s) plants. 

It may be felt that there is nothing to gain by using the scheme of Fig. 
8.2-4 instead of that in Fig. 8.2-5, where the overall plant input and output 
alone are used to drive an estimator. After all, nothing is saved in terms of the 
overall estimator dimension by using the scheme of Fig. 8.2-4. There are 
actually several reasons. First, one or more of the h^C?) may be such that 
there is a pole-zero cancellation between two or more of the w t (s). In this 
instance, it may not be true that measurement of the overall plant output alone 
will allow observation of all the states — i.e., if a composite F matrix and h 
vector are formed for the whole plant, the pair [F, h] may not be completely 
observable. Second, the noise properties associated with the first scheme may 
be better, if only because more information is being used than in the scheme 
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overall 

estimator 



Fig. 8.2-4 Estimator for the scheme of Fig. 8.2-3. 




Fig. 8.2-5 Alternative estimation scheme. 



of Fig. 8.2-5, where some is actually being discarded. It would be intuitively 
reasonable to suppose that if there were almost, but not quite, a pole-zero 
cancellation between two of the small plants, the estimator of Fig. 8.2-5 
would be especially susceptible to noise. Third, depending on the physical 
devices used to implement the estimators, it may be easier to construct the 
individual low-order estimators of Fig. 8.2-4 than the single high-order one 
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of Fig. 8.2-5. Fourth, if any of the small plants have transfer functions of the 
form aj(s + /?j) or (s + oc^/Cs + jSj), estimation of the state can be imme- 
diately achieved, since in the case of the first form, the state is the output 
(or an appropriate multiple), and in the case of the second form, the state is 
a linear combination of the input and output. Accordingly, there is reduction 
in the complexity of the associated estimator. Finally, we shall show in the 
next chapter that if all we really require is k'x e rather than in order, say, 
to implement a feedback law, the scheme of Fig. 8.2-4 allows further drastic 
simplification in that the set of estimators £ ( . may be replaced by a single 
estimator with the same number of inputs as shown, with output k'x e , and 
with dimension equal to the greatest of the dimensions of the E r This is an 
extremely powerful argument in favor of the estimator arrangement of Fig. 

8.2- 4 (or, at least, a modification of it) as opposed to the estimator of Fig. 

8.2- 5. 

It is interesting to note that the scheme of [9] for estimation of the state 
of a multiple-output system relies on representing the system in a certain sense 
as a cascade of smaller subsystems, the inputs and outputs of which are all 
available for measurement. 

We remarked earlier that it is impossible to estimate all the states of a 
plant that are not observable. However, in general, it is still possible to estimate 
the observable components of the states of such a plant; if the unobservable 
components decay extremely fast, this partial estimation may be adequate. 
Problem 8.2-3 asks for an examination of estimator design in these circum- 
stances. 

Problem 8.2-1 : For the system (8.2-17), design a state estimator such that 
F -b k e K has two eigenvalues of —5. 

Problem 8.2-2: Design an estimator for 




y = [ 1 0]x. 



Problem 8.2-3: If a system is not completely observable, by a coordinate 
basis change it may be put into the form 




y = [H\ 0]x, 

where [ F \ is completely observable, see Appendix B. Show how to estimate 

certain of the components of x, and show why it is impossible to estimate the 
remainder. 
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Problem 8.2-4: Devise a computational procedure for estimator design for 
the following system 
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[Hint: Examine K e of the form 

,r*/fyooo<r 
* Lo o o 8 € <f> C- 

where the Greek letters denote elements that are, in general, nonzero.] 

Problem 8.2-5: Consider the scheme of Fig. 8.2-6. Write down state-space 
equations for the individual blocks, and design a state estimator assuming availabil- 
ity of the signals labeled y { and y 2 . Discuss the situation where u and y 2 alone are 
available. 




8.3 NONSTATISTICAL ESTIMATOR DESIGN WITH 
REDUCED ESTIMATOR DYNAMICS 

Our aim in this section is similar to that of the previous section, with a 
few modifications. We assume that a plant is given with the following equa- 
tions: 



x = Fx + Gu 
y = H'x 



(8.3-1) 

(8.3-2) 
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with F, G , and H constant. We seek a system of the form 

w = F e w + G, e u + G le y (8.3-3) 

where F e has lower dimension than F; the vector k>(/) is required to have the 
property that from it and >>(/), a state estimate x e (t) can be constructed with 
x(?) — x e (t) approaching zero at an arbitrarily fast exponential rate. Notice 
that we have required x^/) to depend on the values taken by w and y at time 
t alone — i.e., we require x e {t) to be a memory less transformation (indeed, as 
it turns out, a linear transformation) of w(t) and y(t). The general arrangement 
is shown in Fig. 8.3-1. 




Fig. 8.3-1 General structure of the Luenberger estimator. 

Estimators of this variety were first described in [10] by Luenberger for 
the scalar output case; then the multiple-output case was given in [9]. 
For single-output systems, the idea suggesting that a reduction in estimator 
order might be possible is a very simple one: In an appropriate coordinate 
basis, the plant output constitutes one entry of the state vector. Therefore, if 
the state vector is an n vector, only {n — 1) components really need be esti- 
mated. The generalization for multiple-output systems is also not difficult 
to understand: If there are r independent outputs, then, plausibly, these can 
be used to deduce r components of the state vector when an appropriate 
coordinate basis is used, leaving {n — r) components to be estimated. It 
then becomes possible, although by no means obvious, to estimate these 
remaining ( n — r) components with an (n — r)-dimensional system. The 
difficulty lies in translating these somewhat imprecise notions into an effec- 
tive computational scheme. 

Most of our discussion will be restricted to single-output systems. 
Here, if the plant dimension is n , the estimator dimension is (n — 1). As 
explained in the last section, the assumption that the plant is observable is 
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vital. Thus, we suppose that (8.3-2) is replaced by y — Wx , with [ F , h] a 
completely observable pair. 

We shall now deduce some desirable properties of the estimator, fol- 
lowing which a computational procedure for estimator design will be given. 
Without specifically writing down a coordinate basis change, we can conceive 
a basis change that will have the property that y is the last entry of the state 
vector x in the new coordinate basis. That is, 

y = [ 0 0 - * « 0 \]x. (8.3-4) 



With an overbar denoting the deletion of the last row of a vector or matrix, 
we can then write the plant equations as 
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(8.3-5) 



where the F matrix of the plant has been partitioned conformably with 
the partitioning of x, and where x n is actually y. The matrix F e appearing in 
(8.3-5) is a submatrix of F, and later appears in the estimator equation. 

An estimator of the sort given in the previous section would have an 
equation of the form 
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(8.3-6) 



where k e is chosen so that F + k e h' has desired eigenvalues. If, however, 
is known to be y, there is no point in including a further estimation of 
x„ using x ne , and the last row of (8.3-6) might just as well be deleted. In making 
this deletion, observe also what happens to the last term of the right side of 
(8.3-6): h'x e is just x ne , which is taken to be y from the start. Thus, this term 
disappears, and we are left with 



x e = F e x e + bx ne + Gu = F e x e + By + Gu. (8.3-7) 

In reducing the dimension of the estimator equations, we have unfor- 
tunately lost the effect of the feedback term k e [h'x € — y] of the full-order 
estimator (8.3-6), which, it will be recalled, served to define the rate at which 
the state estimate converged. Equation (8.3-7), the (n — l)th order estimator, 
has no gain vectors, such as k e , that might be adjusted to set the rate of con- 
vergence. However, the matrix F e is adjustable, as we shall note subsequently, 
and we can easily see that this adjustment provides a tool for successful 
estimator design. Let us consider the convergence properties of the estimator 
of (8.3-7); deletion of the last row of (8.3-5) yields 

x == F e x + by + Gu. 



(8.3-8) 
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Subtracting (8.3-7) from (8.3-8) yields 

^(x - x e ) = F e (x - x e ). (8.3-9) 

Consequently, all convergence properties are summed up by the eigen- 
values of F e . Hence, if a coordinate basis is found with the properties that 
(1) y is identical with the last entry of jc, and (2) that the (n — 1) x ( n — 1) 
top left submatrix F e of the F matrix has arbitrary eigenvalues (even though, 
of course, the eigenvalues of the F matrix are invariant with coordinate 
basis change), then we can successfully design an estimator. 

In the remainder of this section, we shall discuss a computational 
procedure leading to a coordinate basis with the desired properties. The 
computational procedure to be presented amounts to revealing an appropriate 
change of coordinate basis. To illustrate how the change of basis is derived, 
we shall, in fact, describe two successive basis changes, each of which indi- 
vidually has some significance. In actual practice, the two basis changes can 
be replaced by a single one. 

We start with the completely observable plant of (8.3-1) and (8.3-2), 
save that H' is a row vector h\ We do not yet assume that h' ~[ 0 0 
0 1] or that the top left-hand (n — 1) x {n — 1) submatrix of /’has certain 
desired eigenvalues. 

First, with det (si — F) = s n + a„s n ~ l + • • • + a l9 define 

a 2 a 3 • • 

@3 Cl 4. ’ Cl 



10 . 

1 0 - • 

This matrix is nonsingular, because [F, h] is completely observable; further- 
more, T i has the property established in the last section that 

0 0 • • 0 

10 * -0 —a 2 

0 1 - * 0 — d ^ 

F t = T t FT r‘= h t — = 

• • • • • 

• • • « * 

_0 0 - - 1 -a n _ 

( 8 . 3 - 11 ) 

We also define G l = 7\(7, but the precise form of this matrix is of no interest. 
Now suppose that the estimator dynamics have been specified, in 
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terms of the eigenvalues of a matrix F ei not yet defined to be a submatrix of 
F. These eigenvalues govern the rate at which x € approaches x. Let the charac- 
teristic polynomial of F e , which is of degree ( n — 1) and not n , be det (si — F e ) 
~ s"~ l + a,,.^” -2 + •••+#!. Define 



“1 0 • 

0 1 * 



0 0 . 
_0 0 . 

Then it is readily verified that 



~1 0 

0 1 



T 




and 



0 0 
0 0 



0 — ocj 

o — a 2 



1 -a,- 1 
0 1 

• 0 a, 

• 0 a 2 



• 1 «,-i 

• 0 1 





”0 


0 • • 


-a, 1 

| 




1 


0 ■ - 


! 

#2 I ^2 




0 


1 


— a., ) b* 


F 2 = T 2 FJV = 


• 


• m 


3 , 3 

1 

* 1 * 

t 

/ 




• 

_0 


• • 
0 • • 


1 . 
1 

1 ! 



where the b, are entries of a vector b given by 
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1 

o 

1 




1 

Q 

1 




R 


a l 




a 2 




a 2 






<*3 




a 3 


» 




m 




• i 


• 




» 










a 

n 




_0 





(8.3-12) 



(8.3-13) 



(8.3-14) 



(8.3-15) 



h'z = h\T J 1 = [0 0 • 0 1], (8.3-16) 

which guarantees that y is the last entry of the state vector. Again, we can 
define G 2 = T 2 G t . 
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The preceding transformations may be interpreted as follows. We 
started with the plant equations 

x — Fx -f Gu y = h'x. (8.3-17) 

By setting x 2 = T 2 T y x, where T % T , is a nonsingular matrix, we obtained 
new plant equations 

x 2 = F 2 x 2 + G z u y = h 2 x 2y (8.3-18) 

where F 2 and h z have the special structures depicted. 

Let us temporarily focus now on the problem of estimating x 2 . [Clearly, 
if we have an estimate x 2g of x 2 , then x e = (T 2 T i )~ 1 x 2e will be an estimate of 
x, approaching x at the same exponential rate as x 2e approaches x 2 .) We 
observe that 




may be derived from the ( n — l)-dimensional system 

x 2e = F e x le + by + G 2 u 



(8.3-19) 

(8.3-20) 



where F e is obtained by deleting the last row and column of F 2 . That is, 



"0 0* • 0 -a, 

1 0 * ■ 0 — oc 2 

0 1 • * 0 —a, 



(8.3-21) 



0 0 



1 -a*-J 



This follows from our earlier discussion; the structure of F 2 given in (8.3-14) 
and the first of Eqs. (8.3-18) guarantee that x 2 satisfies the same differential 
equations as x 2e , and, accordingly, that their difference statisfies 






(8.3-22) 



The matrix F e is the transpose of a companion matrix, and has eigenvalues 
which are the zeros of j *" 1 + oc n - x s n ~ 2 + •..+«,. Accordingly, the state 
estimate converges to x 2 at the desired rate. 

To summarize the computations in the preceding scheme, we can list 
the following steps: 

1 . Compute the characteristic polynomial of F; form the matrix T 1 of 

Eq. (8.3-10). Decide on desired estimator dynamics and form the poly- 
nomial s" ‘ 1 + oc n - l s n ~ 2 + * • • + a,, whose roots govern the rate at 
which x — approaches zero. Construct the matrix T 2 of (8.3-12), 
and transform this coordinate basis, setting F 2 = T 2 T x FT\ x Ti { , etc. 
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2. The matrix F 2 will have the form of Eq. (8.3-14). By appropriate 
partitioning, construct the matrix F e , consisting of the first (n — 1) 
rows and columns of F, and the vector 5, the last column of F except 
for the last element. Define G 2 as G 2 less its last row. 

3. Use F e , 5, and G 2 to implement Eq. (8.3-20). Obtain x 2c as 

* 2 . 
y 

and x, as (' T 2 T i )~ l x le . 

Figure 8.3-2 shows the detailed estimator structure. 





Fig. 8.3-2 Detailed structure of the single-output Luenberger 
estimator. 
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Also, 





There is only one eigenvalue to specify for the matrix F e , Suppose we take it 
at —10. Then 



n ~-io 




which leads to 
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From (8.3-14), the equation for x u is 
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Note that the calculation of F i9 g l9 and h x is not, in fact, necessary. It is suffi- 
cient to find Tj and F 2 ,and proceed straight to F 2 , etc., via F \~T \T X FT\ X T 2 X . 
The plant and estimator are shown in Fig. 8.3-3. 

We shall now give an incomplete discussion of estimators for multiple- 
output plants. A complete discussion can be found in [9]. Even if the plant 
states are observable from one of a number of outputs, it is preferable not to 
employ an observer using just this one output, on the grounds that if more 
output components are used, the dimension of the estimator can be lowered. 
In general, if there are r independent outputs, the estimator need only be of 
dimension n — r, and the greater the number of outputs available, the less 
the dimension of the estimator. 

The case of a plant composed of a cascade of smaller plants, with the 
output of each smaller plant available, is easy to consider using the theory 
for single-output plants. Thus, consider the arrangement of Fig. 8.3-4, where 
n i is the dimension of the small plant /. Estimation of the overall plant state 
may be broken up into estimation of the states of a number of scalar output 
plants, the estimator for the small plant i being of dimension n i — 1. Con- 
sequently, the overall estimator dimension is 2 (+) — r — i.e., r less than the 
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Fig. 8.3-3 Second order plant example with estimator. 




Fig. 8.3-4 State estimation for a special multiple-output plant. 



complexity ° f ° Vera " plam -» h “ h a subs.an.ial saving in 

r ft, A "w the reasons ,isted in the last section in favor of estimating this wav 

herein addition The 613 ' ° Vera11 plant in P ut and output, are applicable 
nere In addition, the complexity of the estimator in this case is simoler 

n the next chapter, we shall discuss how further simplification again of ? the 

estimator structure may be achieved when all that is required is a Lar func- 

ti°n k x ‘ of the components of the state estimator, rather than all components 

separately. r 

We now give qualitative comments on the effect of noise in the Luen- 
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berger estimator. As for the estimators of the last section, it will be noise that 
limits the extent to which the eigenvalues of F e can be made negative, i.e., 
the rapidity with which x e will approach x. But there is a second factor pre- 
sent in the estimator of this section which is absent in those of the last sec- 
tion. Suppose the plant output y includes white noise. Using the scheme of 
the last section, the resulting noise in is bandlimited, essentially because 
there is always integration between y and This is not the case with the 
estimators of this section: To be sure, there will be bandlimited noise in 
arising from white noise in y , but there will also be white noise in arising 
from that in y, because x e is partly determined by a memoryless trans- 
formation on y. Consequently, one might expect the performance of the 
estimators of this section to be worse than those of the previous section in a 
noisy environment. 

Problem 8.3-1 : Consider the scheme of Fig. 8.3-5. Write down state-space 
equations for the individual blocks, and design a Luenberger state estimator assum- 
ing availability of the signals y i and y 2 . 




Fig. 8.3-5 Plant for Problem 8.3-1. 



Problem 8.3-2 : Design a Luenberger estimator for 
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y - [1 1 2]x. 



Problem 8.3-3 : Consider the plant 

TT 

u 

LiJ 

y = l i o]x 

and assume that associated with y is white noise, i.e., noise with power spectral 
density S(co) = a . Design a Luenberger estimator with the F e matrix equal to — oc 
for a positive constant ct. Compute the spectral density of noise in both components 
of x e as a function of (X. [Hint: Obtain the transfer functions relating y to each com- 
ponent of If these are t x {jco) and t 2 (jco ), the spectral density of the noise in the 
components is a 1 tiO’co) | 2 and a | / 20 ‘a>)| 2 .] 
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Problem 8.3-4: Repeat Problem 8.3-3, with the replacement of the Luen- 
berger estimator by a second-order estimator with F + kji possessing two eigen- 
values at -a. Compare the resulting noise densities with those of Problem 8.3-3. 



8.4 STATISTICAL ESTIMATOR DESIGN (THE 

KALMAN-BUCY FILTER) 

In this section, we touch upon an enormous body of knowledge perhaps 
best described by the term “filtering theory.” Much of this theory is sum- 
marized in the two books [11] and [12]. The particular material covered here 
is discussed in the important paper [1 3], although to carry out certain computa- 
tions, we make use of a method discussed in [14] and [15]. 

Broadly speaking, we shall attempt to take quantitative consideration of 
the noise associated with measurements on a plant when designing an esti- 
mator. This means that the design of the estimator depends on probabilistic 
data concerning the noise. We shall also aim at building the best possible 
estimator, where by “best possible” we mean roughly the estimator whose 
output is closest to what it should be, despite the noise. In other words, 
we shall be attempting to solve an optimal filtering , as distinct from optimal 
control, problem. 

We warn the reader in advance of two things: 

1. The treatment we shall give will omit many insights, side remarks, 
etc., in the interests of confining the discussion to a reasonable length. 

2. The discussion will omit some details of mathematical rigor. We 
shall perform integration operations with integrands involving random 
variables, and the various operations, although certainly valid for 
deterministic variables, need to be proved to be valid for random vari- 
ables. However, we shall omit these proofs. Moreover, we shall inter- 
change integration and expectation operations without verifying that 
the interchanges are permissible. 

In outline, we shall first describe the optimal estimation or filtering prob- 
lem — i.e., we shall describe the systems considered, the associated noise sta- 
tistics, and a specific estimation task. Then, by the introduction of new 
variables, we shall convert the filtering problem into a deterministic optimal 
regulator problem, of the sort we have been discussing all through this book. 
The solution of this regulator problem will then yield a mathematical solution 
of the filtering problem. A technique for physical implementation of the solu- 
tion will then be found, leading to an estimator structure of the same form as 
that considered in Sec. 8.2, except that noise is present at certain points in 
the plant, and as a consequence in the estimator, see Fig. 8.4-1. (The figure 




170 STATE ESTIMATOR DESIGN 



Ch. 8 




Fig. 8.4-1 Structure of optimal estimator. 

does not show the structure of the plant, which is assumed to be of the stand- 
ard form x = Fx + Gu, y = H'x , with additional terms in these equations 
representing noise, to be indicated precisely later.) 

Since the structures of the optimal estimator and the estimator of Sec. 
8.2 are the same, we can regard the present section as describing a technique 
for optimally designing one of the estimators of Sec. 8.2. The computations 
required for optimal design are a good deal more involved than those for the 
earlier design for a time-invariant single-output plant. However, for a mul- 
tiple-output plant it is possible that the calculations to be presented here might 
even be simpler than the appropriate multiple-output plant generalization 
of the calculations of Sec. 8.2. The calculations here also extend to time- 
varying plants. 

Description of plants and noise statistics. The plants we shall consider 
are of the form 

^ = F(t)x(t) + G(t)u(t) + v(t) (8.4-1) 

y(0 = H'(t)x{t) + w(t). (8.4-2) 

Here, v(t) and w(t) represent noise terms, which will be explained shortly. 
The dependence of F, G, and H on t, indicated in the equations, is to emphasize 
that, at least for the moment, these quantities are not necessarily time inva- 
riant. However, for infinite-time interval problems, which are considered 
later, we shall specialize to the time-invariant case. Without further comment, 
we assume that the entries of F(-), G(*), and //(•) are all continuous. There 
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is no restriction on the dimensions of u and y in these equations, and the sub- 
sequent calculations will not, in fact, be simplified significantly by an assump- 
tion that either u or y is scalar. 

The properties of the noise terms will now be discussed. First note that 
the model of (8.4-1) and (8.4-2) assumes additive noise only, and it also assumes 
that noise is injected at only two points (see Fig. 8.4-2). The latter restriction 
is not so severe as might at first appear. Thus, e.g., any noise entering with 
u(t) [and passing through the G(t) block] is equivalent to some other noise 
entering at the same point as v(t). 



v(t) w(t) 




Fig. 8.4-2 Plant with additive noise. 

In the case of both v{t) and w(t), the noise is assumed to be white, gaus- 
sian, and to have zero mean. The first property implies that it is uncorrelated 
from instant to instant; if it were also stationary, it would have a constant 
power spectrum. The second property implies that all probabilistic informa- 
tion about the noise is summed up in the covariance of the noise — viz., 
E[v(t)v'(z)] for v(t) and likewise for w(t). This convenient mathematical 
assumption is fortunately not without physical basis, for many naturally 
occurring noise processes are, indeed, gaussian, and such processes normally 
have zero mean. Therefore, in mathematical terms, 

E[v{t)v\z)] = Q(t)S(t - t) E[v(t )] = 0 (8.4-3) 

E[w{t)w\ r)] = R(t)S(t - t) E[w(t)] == 0 (8.4-4) 

for some matrices £>(•) and /?(*)> which we assume without further comment 
to have all entries continuous. The presence of the S(t — r) term guarantees 
the whiteness property. Precisely because the quantities on the left sides of 
(8.4-3) and (8.4-4) are covariances , the matrices Q(t) and R(t) must be 
symmetric and nonnegative definite. But we shall make the additional assump- 
tion that R{t) is positive definite for all r. If this were not the case, there would 
be some linear combination of the outputs that was entirely noise free. Then, 
in an appropriate coordinate basis, one entry of the state vector would be 
known without filtering. As a result, the optimal estimator would not have 
the structure of Fig. 8.4-1 and a different optimal estimator design procedure 
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would be required. We shall omit consideration of this difficult problem here ; 
the interested reader should consult reference f 16]. 

Because it is often the case physically, we shall assume that the noise 
processes v(t) and w(t) are independent. This means that 

E[v(t)w'{ r)] = 0 for all t and r. (8.4-5) 

The final assumptions concern the initial state of (8.4-1). State estimation 
is assumed to commence at some time t 09 which may be minus infinity or 
may be finite. It is necessary to assume something about the state x(/ 0 ). and 
the assumptions that prove of use are that x(/ 0 ) is a gaussian random variable, 
of mean m, and covariance P 0 — i.e., 

E{[x{t a ) - m][x(t a ) - m]’} = P 0 E[x(t a )\ = m. (8.4-6) 

Furthermore, x(/ 0 ) is independent of v(t) and w(t) — i.e., 

£M/o)^(0] = E[x(t 0 )w'(t)] = 0 for all t. (8.4-7) 

Notice that the case where x(t Q ) has a known (deterministic) value is included: 
If P 0 = 0, then x(/ 0 ) = m, rather than just £[*(* 0 )] = m. 

Let us now summarize the plant and noise descriptions. 

Assumption 8.4-1. 

1. The plant is described by the equations (8.4-1) and (8.4-2). 

2. The noise processes v(t) and w(t) are white, gaussian, of zero mean, 
are independent, and have known covariances [see Eqs. (8.4-3), 
(8.4-4), and (8.4-5)]. The matrix R(t) in (8.4-4) is nonsingular. 

3. The initial state of the plant is a gaussian random variable, of known 
mean and covariance [Eq. (8.4-6)]. It is independent of v{t) and w(t) 
[see Eq, (8.4-7)]. 

Notice that an assumption of complete observability of the plant has 
not been made. Such will be made later in considering infinite-time problems. 

Although all the assumptions made often have some physical validity, 
there will undoubtedly be many occasions when this is not the case. Many 
associated extensions of the preceding problem formulation have, in fact, 
been considered, and optimal estimators derived, but to consider these would 
be to go well beyond the scope of this book. 

Statement of the optimal estimation problem. We shall now define the 
precise task of estimation. The information at our disposal consists of the 
plant input u(t) and output y(t) for t 0 <t<t^ and the probabilistic 
descriptions of x(/ 0 ), v(t), and w(t ). To obtain a first solution to the estimation 
problem, it proves convenient to make two temporary simplifications. 

Temporary Assumption 8.4-2. The external input to the plant u(t) is iden- 
tically zero, and the mean m of the initial state x(/ 0 ) is zero. 
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and 



Temporary Assumption 8.4-3. The initial time t 0 is finite. 

These assumptions will be removed when we have derived the optimal 
estimator for the special case implicit in the assumptions. With Temporary 
Assumptions 8.4-2 and 8.4-3 in force, the data at our disposal is simply the 
plant output y(t) for t Q <Ct < t X9 and our knowledge of the covariances of 
<•)> K-X and x(r 0 ). 

Now, since x(t x ) is a vector, it is convenient to regard the problem of 
its estimation as a problem of estimating separately each of its components. 
Therefore, we shall consider the estimation of b'x(t x ) y where b is an arbitrary 
constant vector. Special choices of b , of course, lead to estimates of the various 
components of x(t x ). The sort of estimate for which we shall aim is a minimum 
variance estimate — i.e., we want to construct from a measurement of y(t), 
t*<t <t x , & certain number, call it such that 

E[b'x(t x ) - ft* 

is minimum. Then ft is the minimum variance estimate of b'x(t x ). It turns out 
that because all the random processes and variables are gaussian, and have 
zero mean, the number p can be derived by linear operations on y(t) 9 t 0 < t 
< t l — i.e., there exists some function s(t ; b , t x ) 9 t 0 < t < t x such that 

P = f s’(t;b 9 tjy(t)dt. 

J t « 

(This is a deep result which we shall not prove here.) The introduction of 
s(-;b 9 t x ) now allows the following formal statement of the optimal estima- 
tion problem. 

Optimal estimation problem . Given the plant of Eqs. (8.4-1) and (8.4-2), 
suppose that Assumptions 8.4-1, 8.4-2, and 8.4-3 hold. Then, for fixed 
but arbitrary values of b and t x > t 0 > — oo, find a vector function of 
time s(t ; b 9 O, <t <t u of the same dimension as y(t) 9 such that 

E{[b'x(t x ) ~ £ s’(t;b 9 t x )y(t) rfrj J 

is minimized, the expectation being over all possible realizations of the 
two noise processes and over the random variable x(t 0 ). [A minimum 
variance estimate of b'x{t x ) is then provided by f*s'(t;b 9 t { )y(t) dt\ 

A further problem is to state how this estimate might be physically 
implemented to produce an on-line estimate x e (t\) at time t x of x(tj), which 
is continuously updated, rather than a single estimate of the scalar random 
variable b'x(t x ) for fixed b and t x . 
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Without further comment, we shall use the notation s{t) and s(*) as shorthand 
for s(t; b, /,) and s(> ; b , / t ), provided no ambiguity occurs. 

At this stage, let us pause to review what we have done so far, and what 
we shall do in the next part of this section. Thus far, we have accomplished 
the following: 

1. We have described the plants considered, together with the noise 
associated with the plants. 

2. We have posed a problem of estimating a particular linear com- 
bination of the entries of the state vector at a particular time [i.e., 
b'x(t\) for fixed b and f,], using input and output measurements till 

The estimate is to be a minimum variance one. 

3. We have posed the problem of constructing a device which at every 
time t x will produce an estimate of x(t t ), rather than c^). That is, 
we have posed the problem of constructing an on-line estimate of 

*<>,)• 

In the remainder of this section, we shall do the following: 

1. We shall show how the first of the preceding optimal estimation 
problems can be reformulated as an optimal control problem. We 
caution the reader that the existence of such a reformulation is probably 
not intuitively reasonable , and the parameters appearing in the regulator 
problem are only suggested by hindsight . Therefore, the reader will 
have to suppress such natural questions as “Why pick such-and-such 
set of system equations?” and remind himself that the justification 
for the choice of such-and-such system equation lies in the fact that 
it works, somewhat surprisingly. 

2. Using the optimal regulator problem reformulation, and our knowl- 
edge of the general regulator problem solution, we shall solve the 
specific optimal regulator problem. 

3. Next, we shall do a natural thing — use the solution of the regulator 
problem to write down a solution of the optimal estimation problem 
associated with estimating b'x(t } ) for specific b and 

4. We shall then show how to obtain an estimate of jc(r,) on line. 

5. Elimination of restrictive assumptions, examples, and some exten- 
sions will then follow. 

Reformulation of the optimal estimation problem as an optimal regulator 
problem* To carry through the reformulation, we introduce a new vector 
function of time r(-), of the same dimension as x(-). This function is defined 
from £(•) via the equation 

= -F'(t)r(t) + H(t)s(t ) /-(/,) = b. (8.4-8) 

Observe that Eq. (8.4-8) has the same form as the equation x — Fx + Gu, 
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with prescribed boundary condition x(f 0 ), except that we shall be interested 
in the solutions of (8.4-8) for t < t x rather than for t>t x . 

We shall now rewrite E[b'x(t ,) — s'(*M0 dt] 2 as a quadratic perfor- 

mance index involving r(-) and From (8.4-1) and (8.4-8), we have 

~ = rV)x(t) + r’Wf) 

= — r'Fx + s'H'x + r'Fx + r'v 
= s 'y — + r'v. 

Integrating this equation from t Q to f l9 and using the boundary condition 
on r, leads to 

b'x(t x ) — /*'(/ 0 W*o) ~ f — f £(0^(0 dt -f f r'(t)v(t)dt 

J to J to J to 

or 

b'x(t x ) — f s\t)y{t)dt = r'(/ 0 )4/o) — f s'(/)w(0 ^ + f dt. 

J to J *0 J f 0 

(8.4-9) 

The next step is to square each side of (8.4-9) and take the expectation. 
Because of the independence of x(/ 0 ), w{t), and v(t), there results 

E b'x(t ,) — \ ' s'(t)y(t)dt = £{[r'(f 0 )*(to)] 2 } 

J t 0 

+ E^j‘‘ s’{t)w(t) dtY^ 

+ £ {[J ( ' r VM Odtj J- 

The three expectations on the right side of this equation are easy to 
evaluate, assuming interchangeability of the expectation and integration opera- 
tion in the case of the second and third. Thus, 

E[[rXtMt«W) = E[r'{t 0 )x{t 0 )x'{t 0 )r{t 0 )] 

= r’(t 0 )E[x(t 0 )x'(t 0 ))r(t 0 ) 

= r\t 0 )P 0 r{t 0 ). 

Also, 

j = | s\t)w{t)w\x)s(x) dt dx 

== f f sX0£M/)h'Xt)Mt) dt dx 

J to J to 

= f ( s\t)R(t)5{t — t)s(t) dt dx 

J to J to 




s\t)R(t)s(t) dt. 
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Similarly, 



Therefore, 



E Iff 1 r\i)v{t) 
LJ t 0 




[' ' r'mmt) dt. 



E 




s f (t)y(t) dt 



n 



= r'(t a )P 0 r(t 0 ) + f [s\t)R(t)s(t) + r'{t)Q{t)r{t)] dt. (8.4-10) 

J to 

Now all quantities on the right of (8.4-10) are deterministic, with s(-) free to 
be chosen and r( •) related to •) via (8.4-8). Therefore, the problem of choos- 
ing £(•) to minimize the left side of (8.4-10) is the same as the deterministic 
problem of choosing *?(•) to minimize the right side of (8,4-10), subject to 
(8.4-8) holding. 

Let us summarize the reformulation of the optimal estimation problem 
as follows. 



Reformulated optimal estimation problem. Suppose we are given the 
plant of Eqs. (8.4-1) and (8.4-2), and suppose that Assumptions 8.4-1, 
8.4-2, and 8.4-3 hold. Let b be an arbitrary constant vector and t x > t 0 
be an arbitrary time. Find a function s(t; 6, t x ) for t 0 < t < t x such that 
the (deterministic) performance index (8.4-10) is minimized, subject to 
Eq. (8.4-8) holding. 



We shall now comment upon this reformulated problem. Recall that 
R is positive definite for all t\ Q is nonnegative definite for all t\ and P 0 , 
being the covariance of a vector random variable, is also nonnegative definite. 
Therefore, the only factors distinguishing the problem of finding the function 
^(•), which minimizes (8.4-10) subject to (8.4-8), from the usual optimal 
regulator problem are (1) that the boundary condition on r(t) occurs at the 
final time t { rather than the initial time r 0 , and (2) that the term r\t 0 )P 0 r(t 0 ) 
represents an initial rather than a final value term. In a sense, the problem 
we face here is a regulator problem with time running backward. We shall 
solve the minimization problem by reversing the time variable with a suitable 
transformation. This will give a solution to the minimization problem in feed- 
back form, i.e., ^(/) will be expressed as a certain (linear) function of r(t). 
Then, use of the differential equation for r{t) will allow us to write down an 
explicit formula for s(/). 

Solution to the regulator problem in feedback form. We shall now 
compute the solution of the minimization problem associated with (8.4-8) 
and ( 8 . 4 - 10 ). Define a new running variable i = ~t, and vectors s(t), r(f), 
and matrices F(f), R(i), etc., by s(t) = s(0, r(t) = r(t ), and F{i) = F(t ), 




Sec. 8.4 



STATISTICAL ESTIMATOR DESIGN ( THE KALMAN-BUCY FILTER) 177 



R(i) = R(t), etc., when f = —t. The performance index to be minimized is 

f / 

V = /='('„) Wo) + P WMW) r Ht) QCt)Hl)] di (8.4-11) 

J h 

where f, < f 0 , and the constraint equations are 



zfM = pvm - mm w,) = b. 



(8.4-12) 



Minimization of (8.4-11) subject to (8.4-12) is achieved as follows. Let 
P(t) be the solution of 



d - 

_ p 

di 



A A 



PF' + FP - PHR-'H’P + Q P(i 0 ) = P 0 (8.4-13) 



where the argument i of the matrices in (8.4-13) is suppressed. Then the con- 
trol law 



s(i) - R~\i)H\i)P(i)r{t) t x < i < i ( 



(8.4-14) 



achieves the minimization. Because R(i) is positive definite, and Q(i) and P 0 
are nonnegative definite, it follows from the earlier regulator theory that P(f) 
exists for all t < i 0 and is symmetric nonnegative definite. Now, with the 
definition P{t) ~ P(t) when i ~ — r, Eq. (8.4-13) is readily shown to imply 



dP 

dt 



PF r +FP-PHR' i H , P+Q jP(/ 0 ) = P 0 



(8.4-15) 



where the suppressed argument is now t. Consequently, (8.4-14) implies 

s(t) - R-'(t)H'(t)P(*)r(t) (8.4-16) 

and we conclude that the function j(*) minimizing (8.4-10), subject to 
(8.4-8), satisfies Eq. (8.4-16), with P defined by (8.4-15). We observe, too, 
that the existence of P(t) as the solution of (8.4-15) is guaranteed for all 
t >t Q , because the existence of P(t) is guaranteed for all f < f 0 ; P(/), of course, 
is also symmetric nonnegative definite. 



Explicit solution of the optimal estimation problem. To complete the 
solution of the optimal estimator problem, we must evaluate the function 
s(t) as an explicit function of time. Inserting (8.4-16) into (8.4-8), we have 

^ dt) = ~[F(0 - P{t)H{t)R'\t)H\t))'r{t) r(t,) = b. (8.4-17) 

Let us define 

F/0 = F(t) - (8.4-18) 

and let 3> c (f, r) be the transition matrix associated with x = F e x. Then the 
solution of (8.4-17) can readily be verified to be 

r(t) = t)b. 
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Thus, from (8.4-16), 

s(t) ■■= R- l (t)HXt)PQ)W'(t u t)b . (8.4-19) 

The manipulations between Eqs. (8.4-16) and (8.4-19) serve two pur- 
poses: (1) to introduce the matrix F£t); and (2) to express the function s(-) 
minimizing (8.4-10) subject to (8.4-8) as an explicit function of time, rather 
than as a feedback law. 

Since the solutions to the optimal estimation problem and reformulated 
optimal estimation problem are the same, we shall merely summarize the 
solution to the former. 



Solution to the optimal estimation problem . Given the plant equations 
(8.4-1) and (8.4-2), suppose that Assumptions 8.4-1, 8.4-2, and 8.4-3 
hold. Then for fixed but arbitrary values of b and t x > / 0 , the function of 
time j(r; b, /,), t 0 < / < t l9 which minimizes 



e| b'x{t x ) - £ s'(t; b, t [MO dt J, 



is defined as follows. Let P(t) be the solution of the Riccati equation 
(8.4-15), which exists and is symmetric nonnegative definite for all 
t > t 0 . Let F£t) be defined via (8.4-18), and let <P£t, t) be the transition 
matrix associated with F£t). Then s(t;b,t x ) is given by Eq. (8.4-19), 
repeated for convenience: 



s{t; b, t x ) = R-\t)H f (t)P(t)Q> f e {t x , t)b. (8.4-19) 



Construction of an on-line optimal estimate. We shall now examine the 
problem of physically implementing the estimation procedure on line. Since 
the minimum variance estimate of b'x(t x ) is b' J'' O e (L, i)P{i)H{t)R'\t)y{t) dt 
for arbitrary b , it follows by taking b' — e\ (a row vector consisting of zeros, 
save for unity in the /th position) that the minimum variance estimate of the 
/th entry of x(t x ) is the /th entry of /'; O ;(/,, t)P(t)H{t)R-\t)y(t) dt. In other 
words, 

x£t x ) - f 1 ®.(/ 1 , t)P(t)H(t)R-'(t)y<j) dt. (8.4-20) 

J tQ 

On-line implementation of this estimate is now simple, for observe that 
(8.4-20) is equivalent to requiring 

~ x e (t) = F e (t)x e (t) - K(t)yQ) x e (t 0 ) = 0 (8.4-21) 

where 

K£i) = - P{t)H{t)R~\t ). (8.4-22) 

Figure 8.4-3(a) shows a realization of this equation, which we reiterate is 
valid, provided that u(t) = 0, E[x(t 0 )] = m = 0, and t 0 is finite. If we observe 




Sec. 8.4 



STATISTICAL ESTIMATOR DESIGN (THE KALMAN-BUCY FILTER) 179 




Fig. 8.4-3 (a) First estimator structure [w(r) = 0, m — 0]; (b) 
second estimator structure { u(t) — 0, m ~ 0]. 

from (8.4-18) that F e (t) = F(t) + K£t)H'{i), it follows that (8.4-21) is equiva- 
lent to 

xXO = F(t)x e (t) + K e (t)mt)xJj) - yU)} x e (Q = 0. (8.4-23) 

Figure 8.4-3(b) shows the associated estimator realization. 

We have now covered the basic optimal estimation problem, and our 
task for the remainder of this section consists in tidying up some loose 
ends, and presenting examples. Here, in order, is what we shall do. 

1. We shall eliminate the restrictive Assumption 8.4-2, which required 
a zero plant input and zero mean initial state — the resulting change in 
the estimator is very minor. Then we shall present a simple example. 

2. We shall present an interpretation of the matrix P(t ), the solution of 
the Riccati equation (8.4-15), as a measure of the goodness of the 
estimate at time t , and we shall present a second example exhibiting 
this interpretation. 

3. We shall show how to cope with the case t Q = — oo, drawing special 
attention to time-invariant problems, and including one example. 

Elimination of Assumption 8.4-2. We wish to consider situations where 
u(t) can be nonzero, and £[x(/ 0 )] = m can be nonzero. The effect of nonzero 
values of either of these quantities will be to leave the plant state covariance 
and output covariance the same as before, but to change the mean of the 





180 STATE ESTIMATOR DESIGN 



Ch.8 



plant state and plant output. Thus, since from (8.4-1), 

x(t) = <D(r, t 0 )x{t 0 ) + f $0, r)G(z)u(r) dr + [ <£(>, r)v(z) dr, 

J t 0 J to 

it follows that now 

£[*(/)] = $(/, t 0 )m + f <!>(/, r )G(r)u(z) dr . (8.4-24) 

J to 

A detailed analysis shows that E[x e (t)\ will have the same value as E[x(t)] if 
each entry of x e (t) is 1° be a minimum variance estimate of the corresponding 
entry of x{t ), but otherwise the covariance of x e (t) is unaltered. The way to 
achieve this is to make a modification to the estimator equations (8.4-21) 
and (8.4-23). (The required modification might even have been guessed from 
Chapter 8, Sec. 8.2.) Without carrying out a somewhat complicated deriva- 
tion, we merely state the modified equations: 

*.(0 = F e (t)x e (t) - K e (t)y{t) + G(t)u(t) x e (t 0 ) = m (8.4-25) 
or 

f t x e {i) = F(t)x(t) + K&i)[H'(t)x e {t) - y(t)} + G(t)u(t) x e (Q = m, 

(8.4-26) 

where, as before, K e (t) is given by (8.4-22) and P(t) satisfies the Riccati equa- 
tion (8.4-15). Figure 8.4-4(a) shows plant and estimator according to (8.4-25), 
and Fig. 8.4-4(b) shows plant and estimator according to (8.4-26). Part of 
the estimator in Fig. 8.4-4(b) is enclosed in dotted lines, to emphasize the 
fact that the estimator is a model of the plant with additions. Let us now 
summarize. 

Physical realization of the optimal estimator (t 0 finite ). Given the plant 
equations (8.4-1) and (8.4-2), suppose that Assumptions 8.4-1 and 8.4-3 
hold. Then an on-line estimate x e (t) of x(/) is provided (at time t) by the 
arrangement of Fig. 8.4-4(a) [or 8.4-4(b)], where K&t) is defined via 
(8.4-22) and P(t) via (8.4-15). The matrix P(t) exists for all t > / 0 , and is 
symmetric nonnegative definite. 

By way of example, we consider the following problem. The plant is 
time invariant, with transfer function 1 j(s + 1), and there is input noise of 
covariance 1<5(/ — t), and additive noise at the output of covariance 2 S(t — t). 
At time zero, the initial state has a known value of 1 . The problem is to design 
an optimal state estimator. In state-space terms, the plant equations are 

x ~ — x -\~ u -\- v x(0) = 1 

y = x + w 
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with E[v(t)v( t)] == — t) = l<5(r — t) and £[w(f)w(T)] = R5(t ~ r) = 

2<5(/ — t). The initial state covariance matrix P 0 is zero. From Eq. (8.4-15), 
we have 

P = -2P-|P 2 +1. 

This equation yields 

r> dp i r , 

Jo P 2 +4P- 2 2 Jo 



whence 



r i_ 

_2V 6 



P + 2 - 






- /*(0 1 
.0 = _ 7 ' 



Pit) = jVT ~ 2 )[* — ex P (— q] 

1 + [(V6 - 2)/(VT + 2)] exp t) 

The gain matrix K e (t) for the optimal estimator, here a scalar, is given from 
(8.4-22) as |-P(/). Figure 8.4-5 shows the plant and estimator. 



The matrix P{t) as a measure of goodness of estimation. It may well be 
asked whether P{t) has any significance other than aiding in the computation 
of K e (t)- Indeed it has, as we shall now show. Recall Eq. (8.4-10), rewritten 
for convenience as 

E b'x(t t ) — f s\t)y(t)dt > 

E J t 0 J . 

= rV 0 )P Q r(t 0 ) + f' 1 IsXtWMt) + r’(t)Q(t)r(t)} dt. (8.4-10) 

We recall that we found the function .?(•) minimizing both sides of (8.4-10) 
by viewing the problem of minimizing the right side of (8.4-10), subject to 
(8.4-8), as an optimal regulator problem. We have not bothered to note till 
now what this minimum value is. A quick review of the discussion regarding 
the minimizing of the right side of (8.4-10) will show this minimum to be, in 
fact, b'P(t j) b. Since s\t)y(t) dt is b'xXt,), it therefore follows that 

Pffo'*(/i) - b'xjjy)] 1 } = b'P{t,)b. 

But since b is arbitrary, it is evident that 

£([x(0 - * € (' i )] M ' i ) ~ *.(/,)]'} = Pit,). (8.4-27) 

Thus, P{t x ) is a measure of the error between x(t x ) and x e {t,)\ more precisely, 
it is the error covariance. 

The following example is discussed in [17]. The position and velocity of 
a satellite are to estimated ; the measurements available are, first, a position 
measurement including white gaussian noise, and second, an acceleration 
measurement including a constant random variable measurement error, 
accounting for drift, etc. The motion of the satellite is linear and one-dimen- 
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Fig. 8.4-5 Plant and optimal estimator. 



sional, and there is a constant gaussian random acceleration, independent of 
the measurement error in the acceleration measurement. 

With Xj denoting position and x 2 velocity, the equations of motion are 

*1 = 
x z = a 

where a is the constant acceleration and is a gaussian random variable. Now 
the problem data concerning measurements and noise do not immediately 
allow construction of the standard system equations; therefore, we proceed 
as follows. The measurement of acceleration is y 2 — a + b, where b is a 
constant gaussian random variable. Then 



x . = 



snNj* + (snbp- 



where a a = E[a 2 } and a b = £[£*]. Observe that [atl(at + a ;)]y 2 and 
[at /(at -f <rl)]y z — b are independent, for 



T rr 2 “ir -11 

~ *]1 



Gf+y e< ^ 



at at 



ixi (at + at) 



at + a. 



= 0 - 

So this equation is of the form 



X 2 = M + x 3 . 
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where u is known and independent of x 3 . Moreover, x 3 is a constant gaussian 
random variable whose variance is easily checked to be [ololl(al + ffj)], 
a quantity that we shall call p from now on. 

The full system equations thus become 



"V 




'0 


1 


0” 


~X t 




"0" 


x 2 


— 


0 


0 


1 




+ 


1 


_*3_ 




_0 


0 


0_ 


_*3J 




0_ 



y — [1 0 0]x + n\ 

We assume that u is known and that the initial values of x A and x 2 are known, 
whereas f'fx^O)] = />, and = r8{t — t). 

The Riccati equation becomes 





'0 


0 


0" 




0 


1 


0" 




T 


^3- 

II 

^3 


1 


0 


0 


+ 


0 


0 


1 


p - p 


0 




_0 


1 


0 




0 


0 


0j 




_0_ 



r l [ 1 0 0 ]P 



with initial condition 

0 0 0 

/>((>) = 0 0 0 
.0 0 Pi 

The solution to this equation turns out to be 

r P_ li 

4 2 

T t2 

£ t 



m = 



r 5 / 20 + rjp 



and the optimal gain vector is 

t 4 



k' = - 



t 2 

2 



4(t 5 /2Q + rip) 2(t 5 /20 + rjp) 2(/ 5 /20 + rjp)] 



In the limit as t — > oo, P{f) — 0 and k e (t) = 0. Essentially what happens 
is that x 3 is exactly identified as / — > oo. Since Xj(0) and x 2 (0) are both known, 
this means that x x and x 2 become exactly identified as / — ► oo. In consequence, 
the error covariance approaches zero— i.e., Pit) -> 0, Simultaneously, the 
noisy measurements become of no use, and thus k € {t) — ► 0. 



Initial times in the infinite past. The interpretation of P(t) as the error 
covariance will now be used in a discussion of the estimation problem for 
t 0 = —oo. At this stage therefore, we drop Assumption 8.4-3, but introduce 
a new one. 



Assumption 8.4-4. For all /, the pair [F(t), H(t)] is completely observable. 
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To get a rough idea of the reason for this assumption, consider a time- 
invariant plant having all unobservable, unstable states. Suppose also that at 
time t 0 — oo , the initial state of the plant has mean zero. Since any estimator 
can deduce absolutely no information about the plant state from the available 
measurements, the only sensible estimate of the plant state is x e (t { ) = 0. 
Now the covariance of the plant state vector will be infinite at any finite 
time, and therefore so will the error covariance. Consequently, if P(t) retains 
the significance of being the error covariance in the t Q = — oo case, it is 
infinite. The gain K e of the optimal filter will certainly have some, if not all, 
entries infinite also. The purpose of Assumption 8.4-4 is to prevent this 
sort of difficulty. In fact, under this assumption, and, to simplify matters, 
under the assumption P 0 = 0, we claim that P(t) exists as the solution of 
(8.4-15) (and is finite, of course) when (8.4-15) has the boundary condition 
lim P(t 0 ) = 0. A formal justification of this result can be achieved as follows. 

With t = — t, F(i) = F(t), and H(i) ~ H(t ), the complete observability of 
[F(r), H(t)\ implies the complete controllability of [F\t\ H(t ) ]. Considering 
the regulator problem associated with (8.4-11) and (8.4-12), we see by our 
earlier results on the infinite-time regulator problem that the existence of a 
solution P to the Riccati differential equation (8.4-13) is guaranteed under the 
boundary condition lim P(t Q ) = 0. Since P(i) = P(t ), the claim is established. 

If F and H are constant (i.e., the plant is time invariant) and if Q and R 
are constant (i.e., the noise is stationary), it follows (again, by examining the 
dual regulator problem) that the value of P{t) obtained by letting t 0 — > — oo 
is independent of time, and can therefore be computed by evaluating lim Pit) 

where P(t) satisfies (8.4-15) with the initial condition P(0) = 0. Also, the 
constant matrix P is a solution of the quadratic matrix equation 

PF f + FP — PHR~ l H'P + Q = 0. (8.4-28) 

The gain of the optimal estimator is then constant, being given by 

K e = - PHR r 1 (8.4-29) 

and if G is constant, the optimal estimator is a time-invariant system: 

= Fx e +Gu + K e [H'x e - y]. (8.4-30) 

However, this equation is of little importance from the practical point of 
view unless it represents an asymptotically stable system or, equivalently, 
F e + K e H' has eigenvalues with negative real parts. The way to ensure this 
is to require the following. 

Assumption 8.4-5. With constant Fand Q > and D any matrix such that DD ' 

= Q , the pair [F, D ] is completely controllable. 

One way to prove the asymptotic stability of the optimal estimator 
under this assumption (with also G , H , and R constant, t Q — — oo), is to 
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examine the associated regulator problem and to apply the results of earlier 
chapters. The assumption serves to provide an observability assumption for 
the regulator problem, which guarantees asymptotic stability for the optimal 
regulator. This carries over to the optimal estimator. 

For the time-invariant plant, stationary noise problem, if Assumptions 
8.4-4 and 8.4-5 hold, it can also be shown that P is positive definite and is the 
unique positive definite solution of (8.4-28). Since the time-invariant problem 
is so important, we shall summarize the results. 



The time-invariant optimal estimator . Given the plant equations (8.4-1) 
and (8.4-2), suppose that F , G , and H are constant. Suppose also that 
Assumptions 8.4-1 and 8.4-4 hold, that Q and R are constant, and that 
t 0 = — oo. Then the matrix P, which is the error covariance, is constant 
and satisfies (8.4-28); it may be computed by taking a limiting boundary 
condition of the form limP(f 0 ) — 0 for (8.4-15) and evaluating P(t) 

t a — ► — oo 

for any t, or by evaluating lim P(t) where P{t) satisfies (8.4-15) but with 

*OQ 

boundary condition P(0) = 0. The optimal estimator is time invariant. 
Moreover, if Assumption 8.4-5 holds, the optimal estimator is asymp- 
totically stable. 



The result is, of course, closest to the ideas of Sec. 8.2. There, we confined 
our attention to time-invariant systems, and time-invariant estimators. 
There, also, the estimators were asymptotically stable, as a result of the com- 
plete observability assumption. 

Let us now consider an example. We suppose white noise of covariance 
gS(t — t) is the input to a plant of transfer function l/s(s + l), starting from 
time t Q ~ ~ oo. There is additive output noise of covariance rd(t — t). 
Thus, we take 




Figure 8.4-6(a) shows the scheme. To put this in the standard form, we set 



E[v(t)v'( t)] 



q O' 
0 0_* 



The input noise is not available to the estimator as an input. Denoting by 
Pij the entries of the P matrix, the quadratic equation (8.4-28) becomes 



p 11 


Pl 2 
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P 12 
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It follows that p 12 = a frq from the first equation, p iz = r[V 1 + 2^/qjr — 1] 
from the third equation, and p iX = V rq + 2g from the second equation. 
In solving the third equation, only the positive solution for p 22 is acceptable, 
since P must be positive definite. The estimator gain from Eq. (8.4-29) is 



If, e.g.,# 
or 



-# 

Vl + 2 a /qjr + 1 
16, r = 1, we have k' e = [—4 —2]. The matrix F e is i 7 + k e h\ 



'0 —4“ 

_1 -3_* 

It is readily checked to have eigenvalues with negative real parts. The plant 
and estimator are shown in Fig. 8.4-6(b) for the case q = 16, r = 1. 

We conclude this section with two comments. First, an optimally 
designed estimator may be optimal for noise covariances differing from those 
assumed in its design. This holds for precisely the same reason that a control 
law resulting from an optimal design may be optimal for more than one 
performance index, a point we discussed earlier. 

Second, in the interests of achieving an economical realization of an 
estimator, it may be better to design a suboptimal one that is time invariant, 
rather than an optimal one that is time varying. For example, suppose the 
plant whose states are being estimated is time invariant, and that estimation is 
to start at time zero. Suppose also that at this time, the initial state of the 
plant is known to be zero. Then there would not normally be a great deal of 
loss of optimality if, instead of implementing the optimal estimator which 
would be time varying, a time-invariant estimator were implemented, designed 
perhaps on the assumption that the initial time t 0 were — oo and not zero. 
Reference [8] obtains quantitative bounds on the loss of optimality incurred 
in this arrangement. 



Problem 8.4-1. Consider the plant x = x + v, y = + w 9 with E[v(t)v(x)] 

= E[w(t)w (t)] = S(t t) and v and w independent. Suppose that at time zero, 
x(0) is known to be zero. Design an optimal estimator. 



Problem 8.4-2. Repeat Problem 8.4-1 for the case when the initial time t Q 
is — oo. Also with t 0 = — oo, suppose that an additional measurement becomes 
available, i.e., suppose now 



y = 




L.y 2 J 



where y x — x -f w i9 y 2 = x + w 2 and Fbvj^w^T)] 



E[w 1 (t)w 1 (r)] = S(t — t) 
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with Vy Wi and w 2 independent. Design an optimal estimator for this case. Compare 
the error covariances for the single and multiple output cases. 



Problem 8.4-3, Suppose that x — ax 4- v 9 y x = x + w u y 2 = x + w 2y 
where v, w i, and w 2 all are independent, with covariances q5(t — t), r u S(t — t), 
and r 22 d(t — t), respectively. For the t 0 = — °° case, derive analytic expressions 
for the error covariance, assuming that alone is available and that both y x and 
y 2 are available. 

Problem 8.4-4. Given x = Fx + Gu + v y y = H'x + w, with F , G, and H 
constant and v and w stationary white noise, suppose that f 0 is finite. Show that the 
estimator will be time invariant if £[x(f 0 )*'(fo)] takes on a particular value-in 
general, nonzero. 

Problem 8.4-5. Consider the standard estimation problem with u(t) = 0, 
£[x(/ 0 )] = 0, and t 0 finite, save that ^( •) and w( •) are no longer independent : rather, 
E[v(t)w'(' T)] — S(t)S(t — t) for some matrix S(t). Show that the problem of finding 
a minimum variance estimate of b'x(t 0 for arbitrary constant b and arbitrary 1 x 
is again equivalent to a quadratic regulator problem, with a cross-product term 
between state and control in the loss function. Attempt to solve the complete estima- 
tion problem. 



Problem 8.4-6. (The smoothing problem.) Let x e (t 0 1 / ,) denote the minimum 
variance estimate of x(f 0 ), given measurements up till time t x > t 0y where, as 
usual, i = Fx + Gu + v, y = H f x + w, E[v(t)v'(T)] = Q(t)5(t - r), E[w{t)w'(T)] 
= R(t)S(t — t), = P 0 , £[*o] = and v, w, and x(t Q ) are independent 

and gaussian, the first two also being zero mean. It is desired to define a procedure for 
computing *,(r 0 1 / 1 ). 

Consider the scheme of Fig. 8.4-7. Observe that x Xe (t (/) = x e {tj^\ t) and 
x 2e {t\ 0 — x e (t 0 | r), implying that the smoothing problem may be viewed as a 
filtering problem. Show that 



where 



= (F - Pi x HR~'H')x le + P, i HR-'y x Xe (t Q \t 0 ) = m 
*ie = -P\ 2 HR-'ff'x le +F' l2 HR-'y x le (t,\t Q ) = m 



Pn =Pi,F' + PPu -P U HR^H'P X1 + Q Pn(/o)=Po 
P12 = (F— PM“*/rOPl2. • Pl 2 (/o) = Po 



Show also how to find x e (t 2 \t l ) for arbitrary / 2 <t x , [Hint: Represent the system 
of Fig. 8.4-7 as a 2n-dimensional system with F matrix F a , etc., and let 




Pxi 
P 22- 



be the solution of the associated filtering Riccati equation. Use P to define the 
optimal 2«-dimensionaI filter, and show that the two differential equations shown 
follow from the optimal filter description. This technique appeared in [15].] 
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Fig. 8.4-7 System augmented with integrators. 



REFERENCES 

[1] Popov, V. M., “Hyperstability and Optimality of Automatic Systems with 
Several Control Functions,” Rev. Roumaine Sci. Tech . — ElectrotechnoL Energy 
VoL 9, No. 4, 1964, pp. 629-690. 

[2] Anderson, B. D. O., and D. G. Luenberger, “Design of Multivariable Feed- 
back Systems,” Proc. IEE, Vol. 114, No. 3, March 1967, pp. 395-399. 

[3] Luenberger, D. G., “Canonical Forms for Linear Multivariable Systems,” 
IEEE Trans . Auto . Control , Vol. AC-12, No. 3, June 1967, pp. 290-293. 

[4] Wonham, W. M., “On Pole Assignment in Multi-Input Controllable Linear 
Systems,” IEEE Trans. Auto. Control , Vol. AC-12, No. 6, December 1967, 
pp. 660-665. 

[5] Tuel, W. G., “On the Transformation to (Phase-Variable) Canonical Form,” 
IEEE Trans. Auto. Control , Vol. AC-11, No. 3, July 1966, p. 607. 

[6] Gopinath, B., and B. O. Lange, “On the Identification and Control of Linear 
Systems,” Tech. Rep . SUDAAR No. 351 , June 1968, Dept, of Aeronautics and 
Astronautics, Stanford Univ., Calif. 

[7] Rosenbrock, H. H., “The Assignment of Closed Loop Poles,” Control Systems 
Centre Rep. No. 20 , October 1967, Control Systems Centre, University of 
Manchester, England. 

[8] Singer, R. A., The Design and Synthesis of Linear Multivariable Systems with 
Application to State Estimation,” Tech. Rep. 6302-8 , June 1968, Stanford Elec- 
tronics Laboratories, Stanford, Calif. 

[9] Luenberger, D. G., “Observers for Multivariable Systems,” IEEE Trans . Auto. 
Control Vol. AC-11, No. 2, April 1966, pp. 190-197. 






Ch.8 



REFERENCES 1 91 



[10] Luenberger, D. G., “Observing the State of a Linear System,” IEEE Trans . 
Military Electron. , Vol. MIL-8, No. 2, April 1964, pp. 74-80. 

[11] Wiener, N., “ Extrapolation , Interpolation and Smoothing of Stationary Time 
Series ,” M. I. T. Press, Cambridge, Mass*, 1949. 

[12] Bucy, R. S., and P. D. Joseph, Filtering for Stochastic Processes and Applica- 
tions to Guidance , Interscience Publishers, New York, 1968. 

[13] Kalman, R. E., and R. S. Bucy, “New Results in Linear Filtering and Predic- 
tion Theory,” Trans . ASME Ser. D: J. Basic Eng., Vol. 83, March 1961, pp. 
95-108. 

[14] Bryson, A. E., and M. Frazier, “Smoothing for Linear and Nonlinear Dynamic 
Systems,” Proc. Optimum Systems Synthesis Conf, USAF Tech. Rep. 
ASD-TDR-063-1 19, February 1963. 

[15] Zachrisson, L. E., “On Optimal Smoothing of Continuous Time Kalman 
Processes,” Inform. Sci., Vol. 1, 1969, pp. 143-172. 

[16] Bryson, A. E., and D. E. Johansen, “Linear Filtering for Time- Varying Systems 
Using Measurements Containing Colored Noise,” IEEE Trans. Auto. Control, 
Vol. AC-10, No. 1 January 1965, pp. 4-10. 

[17] Kalman, R. E., and T. S. Englar, “A User’s Manual for the Automatic Synthesis 
Program,” NASA Contractor Rep. NASA CR-475 , June 1966. 




CHAPTER 




SYSTEM DESIGN USING 
STATE ESTIMATORS 



9.1 CONTROLLER DESIGN— BASIC VERSIONS 



This chapter is concerned with tying together the notions of state- 
variable feedback and estimation. In other words, we consider controllers 
of the sort shown in Fig. 9.1-1. In this section, we describe the basic control- 
lers obtained by using the estimator designs of the previous chapter, and we 
discuss various properties of the overall controlled system. These properties 
include frequency domain characteristics, stability properties, and the carrying 
over of the various properties that optimality implies. The remaining sections 
complement and extend the ideas, principally by pointing out modified ver- 
sions of the controller. 

As our basic plant, we take the time-invariant system 

x = Fx + Gu (9.1-1) 

y = H'x. (9.1-2) 

For the observer, we shall take for the moment the structure of Chapter 8, 
Secs. 8.2 and 8.4 — viz., 

= (F + K e H’)x e + Gu- K e y. (9.1-3) 

We assume that K e is chosen so that all eigenvalues of F + K e H' have nega- 
tive real parts. Whether it is optimal for some noise statistics is irrelevant for 
our considerations here; either the scheme of Sec. 8.2 or that of Sec. 8.4 can 
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u 




Fig. 9.1 -1 The basic controller. 



be assumed to lead to the choice of K e . Subsequently, in this section we shall 
consider the use of Luenberger observers in controller design. 

We further assume that we should like to implement the control law 
u = w ext + K'x (w ext denoting an external input), where, presumably, K 
has been selected as an optimal control law. However, for obvious reasons, 
we implement the following law instead : 

u = u exi +K'x e . (9.1-4) 

Equations (9.1-1) through (9.1-4) sum up the entire plant-controller 
arrangement. We shall now study some properties of the arrangement using 
these equations. We begin by making a slight rearrangement. First, from 
(9.1-1) and (9.1-4), we have 

x = (F + GK')x - GK\x - x e ) + Gu ext . (9.1-5) 
Second, from (9.1-1), (9.1-2), and (9.1-3), we have 

X.) = (F + K e H’)(x - x e ), (9. 1 -6) 



which holds independently of w ext . Now we regard the 2 n vector, whose first 
n entries are x and whose second n entries are x — x e9 as a new state vector 
for the overall plant-controller scheme. (It would, of course, be equally valid 
to take as a state vector a 2 n vector with the first n entries consisting of x and 
the second n entries consisting of x € .) 

The plant-controller arrangement, then, has the following description — 
the first equation following from (9.1-5) and (9.1-6), the second from (9.1-2): 



A 

dt 



x 




~F + GK' 


-GK' ~ 


— 

X 


+ 


G 


0 


F + K e H'_ 


-X - x e „ 


_ 0 _ 



(9.1-7) 



y=[H' 0] 




(9.1-8) 



With input u cxt and output y, the plant-controller arrangement has the fol- 
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lowing transfer function matrix, derivable by manipulating (9.1-7) and (9.1-8): 

W(s) - H'[sl - (F + GK')Y l G. (9.1-9) 

This is exactly the transfer function matrix that would have resulted if true 
state-variable feedback were employed. The poles of the open-loop plant, 
corresponding to the zeros of det (si — F ), are shifted to the zeros of 
det [si — (F + GK')]. The zeros of a scalar W(s) are unaltered. 

Thus, from the steady-state (or zero initial state) point of view, use of the 
estimator as opposed to use of true state-variable feedback makes no differ- 
ence. This is, of course, what should be expected. For the case in which the 
steady state has been reached, x — x e has approached zero and x == x e , 
or, in the case of zero initial state, x = 0 and jc — = 0, so that again 

x — x e . Clearly, with X — x e , the control used is precisely that obtained with 
true state-variable feedback. 

From the transient point of view, the plant-controller scheme of Eqs. 
(9.1-7) and (9.1-8) will behave differently from a scheme based on true state- 
variable feedback. Equation (9.1-7) defines a 2«-dimensional system, whereas 
state-variable feedback yields an n-dimensional system. However, the 2 n- 
dimensional system is still asymptotically stable: inspection of (9.1-7) shows 
that the eigenvalues of F + GK ' and of F + K e H' determine the characteris- 
tic modes. (The eigenvalues of F + K e H ' are, of course, associated with the 
additional new modes, which are evidently uncontrollable.) 

We shall now consider a simple example, stemming from a second- 
order position controller. A servomotor, often through gears, drives an iner- 
tial load with coulomb friction. The transfer function relating the input 
voltage of the motor to the output shaft position is of the form K/s(s + a ), 
and here we shall assume normalization to 1 / 5(5 + 1). This transfer function 
is representable by the state-space equations 





x + 








We are required to choose a control law that minimizes J~ (u 2 + x] + x\) dt , 
and implement it with a state estimator, the associated state estimator poles 

being both at 5 = —3. First, the control law is obtained. In the usual nota- 
tion. 



n 




0 “ 

1 



and R = [1]. We seek P as the positive definite solution of 



PF + F'P - PgR-'g’P + Q = 0. 



By setting 
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and by substituting for F, g, Q, and F, we obtain the equations 

P\i = 1 

Pi 1 Pi 2 PllPzi ~ 0 
2 Pl2 - 2 P 22 — P22 + 1 = 0. 

The first equation gives p l2 , the third /i 22 , and the second p x x . Although there 
are multiple solutions, only one yields the following positive definite P : 

P = 

The optimal control law is now 

u = w ext + k'x 

where k ' = —Fg. Thus, 



2 1 
1 1 



^ ^ext ■^'1 ^2* 

4. 

We must now design an estimator, which will have an equation of the 
form 



x e =(F + k e h')x e +gu - k e y . 



The problem is to choose a gain vector & e so that F + has two eigen- 
values at —3. Using the techniques of the previous chapter, or even by trial 
and error, we derive k f e — [— 2 —7]. This leads to the plant-controller 

arrangement of Fig. 9.1-2. 

We consider now how the ancillary properties of optimality may be 
derived. Rather than proceed exhaustively through an analysis of all of them, 
we shall indicate by considering only some properties the sort of thing that 

can be expected. 

Figure 9.1 -3(a) shows a nominally optimal system (with state estimation 
included in the control arrangement), where a nonlinearity <j>(») has been 
introduced. For the zero external input case (i.e., for consideration of Lya- 
punov stability), the scheme of Fig. 9.1-3(b) is equivalent. The nonlinearity 
<£(•) is assumed to be of the usual sector type — i.e., for a single-input system, 
the graph of the nonlinearity lies strictly within a sector bounded by straight 
lines of slope £ and oo (see Fig. 9.1-4). The appropriate generalization applies 
for multiple-input systems. Denoting the gain of the nonlinearity by p, we 
have \ < P < oo. Also, Eq. (9.1-7) is replaced by (for a single-input plant) 



A 

dt 





~F + pgk' -figk' ~ 


x 


0 F+ K e H'_ 


JX - x c _ 



(9.1-10) 



where the argument of p — viz., k'x — has been suppressed. As before, x — 
decays exponentially. The equation for x yields 



jc = (F + pgk’)x + Pw (9.1-11) 



where w is a vector all of whose entries decay exponentially to zero. By insist- 
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ing that /?</?< co for some constant p, we can guarantee an exponential 

bound on Pw. It turns out, although we shall not prove it, that the asymptotic 
stability properties of 

x=(F + p g k’)x (9.1-12) 

established earlier then serve to guarantee also the asymptotic stability of 
(9.1-11). 

It may well be argued that a more natural place to locate the nonlinearity 
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Fig. 9.1-4 A permissible nonlinearity. 

would be right at the plant input, so that Fig. 9.1-5(a) would aDoIv If th* 

estimator equation is still Fn IQ i 'W *- a c 

4 cuiun is sun tq. (9.1-3), repeated for convenience, 

= ( F + K .H')x e +gu— K e y (9.1-3) 

then, with plant equation 



we obtain 



x = Fx -f- p gu 



(9.1-13) 



dt ^ -( F + K .H')(x - x e ) +(0- 1 ) gu . (9.1-14) 

In this instance, will not longer be a true estimate of x, unless u is zero. 
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Fig. 9.1-5 



Even if u decays exponentially, the rate at which x approaches x e is no longer 
governed solely by the eigenvalues of (F + K e H'). Something is seriously 
wrong, and, in general, the overall plant controller could not be expected to 
be asymptotically stable for all sector-limited /?. 

The only way out of this dilemma is to introduce the same nonlinearity 
into the controller, so that the estimator part of the controller again becomes 
a true model of the plant. [See Fig. 9.1 -5(b).] The estimator equation (i.e., 
equation of the block marked “linear estimator” together with the non- 
linearity) is 

x e = (F + K e H')x e + Pgu — K e y. (9.1-15) 

Now, (9.1-13) and (9.1-15) together yield again 

2 F (X - x t ) = (F + K e H')(x - x e ). (9. 1 -3) 

Therefore, correct estimation now takes place. Consequently, nonlinearities 
of the usual sector type will again be tolerable, in the sense that asymptotic 
stability will be retained. 

However, there is a potential difficulty with this somewhat artificial 
arrangement. If the exact nature of the input nonlinearity to the plant is 
unknown, then it is impossible to construct the estimator. Of course, if the 
nature of the nonlinearity is roughly known, and the nonlinearity included 
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in the estimator approximates that in the plant, then, presumably, perfor- 
mance will be satisfactory. 

As a second and representative property of optimal systems, we choose 
sensitivity reduction to parameter variations. We examine here the way this 
property carries over to the situation where an estimated, rather than an 
actual, state vector is used in implementing the optimal linear feedback law. 

Figure 9.1-6 shows a redrawing of the plant-controller arrangement of 
Fig. 9.1-1, as a unity positive feedback system. The quantity fed back is K'x„ 
which in the steady state becomes K'x. Therefore, the transfer function matrix 
of the block enclosed in dotted lines must be K\sl — F)~'G. This is precisely 
the same transfer function that arises when true state feedback is employed. 
We conclude that the same sensitivity results hold for the scheme of Fig. 
9.1-6 as for the scheme when true state feedback is applied. More precisely, 
the output of the scheme of Fig. 9.1-6 will vary less as the result of a parameter 
change in the block in dotted lines than it would if a cascade controller were 
used to achieve the same transfer function. Moreover, with appropriate restric- 
tions on the parameter variations, all components of the state of the linear 
plant part of the block, and the output of the linear plant, will have lower 
sensitivity to parameter change than the equivalent open-loop system. 




Fig. 9.1 -6 Redrawing of the plant-controller arrangement. 



In Chapter 7, we drew attention to the fact that the use of a feedback 
law achieving sensitivity reduction was tantamount to using an optimal 
feedback law. It might therefore be thought that use of the feedback law 
u = K'x e for the system defined by (9.M) through (9.1-3) might not then be 
merely approximately optimal, but in fact precisely optimal. This conclusion 
is not immediately justified, however, because the system (9.1-7) is not 
completely controllable. Problem 9.1-4 seeks further clarification of this 
point, and thus our discussion of the way properties of optimal systems 
carry over to the new controller arrangement is concluded. 

In the remainder of this section, we shall discuss briefly the equations of 
the plant-controller arrangement when a Luenberger observer is used. We 
restrict attention to single-output systems. We first observe that there is no 
loss of generality involved in describing the plant and controller by state- 
space equations with a special coordinate basis when we are seeking to com- 
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pute the overall transfer function matrix, or the qualitative stability properties 
of the arrangement. Therefore, we may assume that the n X n matrix F of 
the plant in (9.1-1) has the special form of Sec. 8.3, — i.e.. 



F ' b 1 

F=\ ! (9.1-16) 

0 0 ... 0 1 1 

where F e is an (n — 1) x (n — 1) matrix, which is the transpose of a com- 
panion matrix form, and possesses desired eigenvalues. 

Also, 

h f = [0 0 ■ • ♦ 0 1]. (9.1-17) 

As we know from Sec. 8.3, is given from 



with 




(9.1-18) 



Xe = F e x e + by + Gu. (9.1-19) 

The overbar denotes deletion of the last entry or row, as the case may be. 
The control law that is implemented is 



u = w e * t + K'x e 

= w ext + (K)'x e + K'y (9.1-20) 

where K f is the last column of K\ 

The plant-controller arrangement is described by the four equations 
(9.1-1), (9.1-18), (9.1-19), (9.1-20), together with equation y = h'x. A state 
vector for the overall plant-controller arrangement is, therefore, provided by 
the (2 n — 1) vector 

Del 



L*J 

However, a more appropriate, but equally acceptable, choice for state vector 
proves to be 

This results in the following equation: 



d_ 

dt 




together with 



rT+ GK’ 





(9.1-21) 



y = h ’ x ■ (9.1-22) 

These two equations then imply again that the transfer function matrix 
relating u to y is h'[sl ( F ■ GK')]~'G, and that the zero-input response of 
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the plant-controller arrangement is determined by the eigenvalues of F + GK r 
and of F e . The only difference between the Luenberger estimator and that 
considered earlier is that the n X n matrix F + K e H' is replaced by the 
("-!) X (n — 1) matrix F e . The carrying over of the ancillary properties 
associated with optimality continues as before. 

We consider now a brief example. As in the earlier example, we consider 
the plant with transfer function 1 js(s + 1), modeled by 






y = [1 




and we suppose that the control law that has to be implemented is 



I* = *4xt - U l]X 

to ensure optimality. Suppose, also, that the estimator pole is required to 
be at s = — 3. Let us change the coordinate basis to achieve this end. Using 
the methods and notations of Sec. 8.3, we obtain 



T 



i 



1 

1 



r 
o ’ 





y 



and with x 2 = T 2 T x x (rather than x 2 denoting the second component of 
x!), the new state-space equations are 





r 

0 



y = [ o l]x 2 



with u = u ext — [1 3]x 2 . Accordingly, the estimator equation is 

*2e = — “ 6j\ 

The plant-controller arrangement is shown in Fig. 9.1-7, with only the con- 
troller shown in detail. 

References [1] through [3] discuss another class of controllers, which are 
shown in reference [4] to have the same general structure as the controller 
obtained by cascading a Luenberger estimator with a linear control law. 
More precisely, the controller is a linear system with inputs consisting of u 
and y 9 and output the feedback component of u. However, the controllers 
do not actually generate a state estimate x e . Moreover, although in the use 
of a Luenberger estimator the closed-loop transfer function matrix has no 
additional poles introduced in it as a result of using state estimate feedback 
rather than state feedback (because the additional states are uncontrollable), 
the closed-loop transfer function matrix resulting in [1] through [3] does have 
a number of additional poles introduced. For a single-input, single-output 
plant, the number of poles is 2/7 - 1, the sum of the plant dimension and 
controller dimension. Design of the controllers is somewhat akin to the design 
of Luenberger observers; however, minimization of a performance index of 
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Fig. 9.1 -7 Second example of a controller design. 



the form J~ ( w'Rw + [x' z']Q[x' z']') dt is required, where z is a vector con- 
taining various derivatives of u, and w is the pth derivative of u with (p — 1) 
the controller dimension. Some disadvantages of this form of controller over 
that already discussed are as follows: 

1. Large nonlinearities cannot generally be tolerated at the plant input 
(even with adjustments to the controller input). 

2. A higher order Riccati equation must be solved, which may present 
additional computational problems. 

3. Estimates of the plant states are not available. 

4. There is far less control over the additional modes introduced into 
the closed-loop system, and the closed-loop transfer function normally 
possesses more than n poles. 

Chapter 10 discusses the basic problem of optimizing a performance 
index containing derivatives of the control (the use of such performance 
indices for controller design, as noted previously, being only one application). 

A problem is included in Chapter 10 illustrating controller design using the 
ideas of [1] through [3]. 



Problem 9.1 -1 . Consider the plant 




~0 r 
-0 0 _ 



* + 




y = U 



0]x. 



Calculate an optimal control law that will minimize the performance index 
J,~ (u 2 + 2x\ + 3*1) dt, and design controllers of both dimension 2 and dimension 
1 with poles that have real parts with significantly larger magnitude than the closed- 
loop system poles. 
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Problem 9.1 -2. Discuss the gain and phase margin properties of optimally 
controlled systems where a state estimator is used. 

Problem 9.1-3. Consider the first-order plant 

x = x + w. 

The control law u = — 2x is an optimal law for this plant. Design state estimators 
of dimension 1 with poles at —1, —5, —10. Then sketch the response of x{t) given 
x(0) = 1 for the following eight cases. In case 1, no estimator is used, and in cases 
2 through 8 an estimator is used. 

1. The feedback law is u = — 2x. 

2. Estimator pole is at — 1 and x e (/o) = 0. 

3. Estimator pole is at —1 and x<,(to) = h 

4. Estimator pole is at —1 and jc*(t 0 ) = —1. 

5. Estimator pole is at —5 and x e (t 0 ) = 0. 

6. Estimator pole is at —10 and jt c (/ 0 ) — 0. 

7. Estimator pole is at —10 and x e (t 0 ) = 

8. Estimator pole is at —10 and jc € (t Q ) = — 1. 

Comment. 

Problem 9.1-4. Consider the 2«-dimensional system defined by Eqs. (9.1-1) 
through (9.1-3), and let u = K'x be an optimal control law resulting from minimiza- 
tion of a performance index ( u'u + x'Qx) dt. Show that if Q is positive definite, 
there exists a positive definite Q such that u = K'x e is the optimal control for a 
performance index \T q [ uu + i x ' x ' ~ x 'e)Q( x ' x ' ~ x' e )'\dt . Indicate difficulties 
in extending this result to the case where Q is singular. (The conclusions of this 
problem are also valid when Luenberger estimators are used.) [Hint: Take as the 
state vector z, where z' = (x' x' — x' e ).] 



9.2 VARIANTS ON THE BASIC CONTROLLER 

The aim of this section is to indicate some parallels with classical control 
ideas. This will be done by exhibiting some variants on the controller arrange- 
ments of the previous section, the structures of which will be familiar from 
classical control. 

In general terms, we shall derive a controller structure of the form shown 
in Fig. 9.2-1. In classical nomenclature, there is a series or cascade com- 
pensator and a feedback compensator. In Fig. 9.2-2, there is shown a second 
controller structure that we shall develop. However, it is not always possible 
to implement this structure for reasons to be discussed. This is in contrast 
to the first structure, which may always be implemented. 

We now list some general properties of the controller structures. 

1. The controller structures will be derived by manipulations on the con- 
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Fig. 9.2 -1 Controller structure familiar from classical ideas. 




Fig. 9.2 -2 Second controller structure familiar from classical 
ideas. 

troller structure of the last section. These manipulations do not 
affect the input-output steady-state performance (or zero-state per- 
formance) of the overall scheme. In other words, the transfer 
function or transfer function matrix of the overall scheme is still 
H'[sl — (F + GK')]~ { G , where H'(sl — F)~'G is the transfer function 
(matrix) of the plant and u = K'x is the desired feedback law. 

2. There is no restriction to single-input or single-output plants, except, 
perhaps, an indirect one. Since the controller structures are derived 
from controllers of the sort described in the last section, one needs 
to be able to design versions of these for multiple-input, multiple- 
output plants. As we know, the multiple-output case may be hard. 

3. The dimensions of the compensators are the same as the dimension 
of the controllers from which they are derived. This means that if, 
e.g.,' an ^-dimensional single-output system employs an ( n — 1)- 
dimensional controller, then the series compensator and the feedback 
compensator will each have dimension (n — 1). 

4. In view of 3, there are additional modes again in the overall scheme 
beyond those introduced in the controllers of the last section. In 
the case of the controller of Fig. 9.2-1, these additional modes are 
always asymptotically stable. However, this is not the case with the 
scheme of Fig. 9.2-2, which may thus prevent its use. 

We shall now proceed with the derivation of the structures. We start with 
the controller of Fig. 9.2-3. Irrespective of the dimensionality of the controller, 
or its detailed design, there is a linear system between u and K'x e , and a 
second linear system between y and K’x e . In the steady state, the scheme of 
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Fig. 9.2-3 The basic controller. 



Fig. 9.2-3 is, in fact, equivalent to that of Fig. 9.2-4(a). In this figure, transfer 
function matrices are shown for each block. These are inserted to keep track 
of subsequent manipulations. 

Figures 9.2-4(a), 9.2-4(b), and 9.2-4(c) are clearly equivalent. By setting 
^3 ~ U ~ Fig. 9.2-4(d) results. As will be shown, the inverse always 

exists. Figure 9.2-4(d), of course, has the same structure as Fig. 9.2-1. The 
equivalence between Figs. 9.2-4(d) and 9.2-4(e) follows by setting W 4 (s) 
= Figure 9.2-4(e) has the same structure as Fig. 9.2-2. 

Thus, subject to establishing the existence of the inverse of / — W n 
the claims regarding the new controller structures have been established as 
far as their steady-state performance is concerned. What has yet to be checked 
is that the additional modes introduced have acceptable stability properties. 
To do this, and to check that the inverse exists, we now become specific. 
We start by considering estimators in Fig. 9.2-3 with the equation 

i, = (F + K e H')x e + Gu- K e y . (9.2-1) 

Of course, F T K e H f is assumed to have eigenvalues with negative real 
part. From (9.2-1), it follows immediately that the transfer function matrix 
relating u to K l x e is 

W x (s) = K'[sl - (F + K e H')]~ x G (9.2-2) 

and the transfer function matrix relating y to K'x e is 

W 2 (s) - -K'[sl - (F + K e FF)Y'K e . (9.2-3) 

Forming /— W t (s\ we can check the following formula for the inverse 
[/ — JF^s )]" 1 (which clearly establishes, at the same time, existence of the 
inverse): 

W y (s) = J + K'[sl - (F + A;//' + GK’)Y l G. (9.2-4) 

Finally, the transfer function matrix W A {s ), given by W^(s)lV 2 (s), can be com- 
puted to be 

W 4 (s) - -K’[sl - (F + K e H' + GK')Y x K e . (9.2-5) 

[Problem 9.2-1 asks the student to verify the formulas for W 2 (s) and )F 4 (s).] 





System W a (s) 




Fig. 9.2-4 Various system equivalences. 



2 
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Let us now study the state-space equations of the scheme of Fig. 9.2-4(d), 
which will lead to conclusions regarding stability. We adopt the notation x 2 
for the state of the linear system with transfer function W 2 (s), and x 3 for the 
state of the linear system with transfer function W 3 (s). 

Now, the input to the W 2 (s) block is simply H'x. Therefore, this block 
may be described by [see (9.2-3)]: 

x 2 - (F + K e H')x 2 - K e H f x. (9.2-6) 

The output of the lV 2 (s) block is, again from (9.2-3), K'x e . From the block 
diagram it is then evident that the input to the W 3 {s) block is u ext + K'x 2 . 
Therefore, it is described by [see (9.2-4)] 

*3 = (F + K e H' + GK')x 3 + Gu txX + GK'x 2 . (9.2-7) 

The output of the W 3 (s) block is, again from (9.2-4), its input plus K’x 3 , 
i.e., + K'x 2 + K'x 3 . This is the input to the plant. Therefore, the plant 

equation is 

x - Fx + Gu &xt + GK'x 2 + GK'x 3 . (9.2-8) 



Equations (9.2-6) through (9.2-8) constitute a set of state-space equations 
for the scheme of Fig. 9.2-4(d). An equivalent set is provided by retaining 
(9.2-6) and (9.2-8) and forming an equation for x — x 2 — x 3 . This equation 
follows at once from (9.2-6) through (9.2-8), and is 

i - x 2 - x 3 = (F + K e H')(x - x 2 - x 3 ), (9.2-9) 

Equations (9.2-6), (9.2-8), and (9.2-9) combine to give 




'F+K e H ' —K e H f 0 

0 F+ GK ' ~~GK f 

0 0 F + K e H ' j 

roi 



+ G w ext 
0 




(9.2-10) 



The block triangular structure of the square matrix in (9.2-10) guarantees 
that the system modes are determined by the eigenvalues of F + GK' and 
by those of F -f- K e H' counted twice. Consequently, the additional modes 
introduced beyond those of the basic controller of Fig. 9.2-3 (whose modes 
are determined by the eigenvalues of F + GK' and F + K e H') are the eigen- 
values of F + K e H '. These modes are, of course, known to be asymptotically 
stable, and even under the designer’s control. The validity of the proposed 
control arrangement is now established. 

Two interesting points follow from Eq. (9.2-10). First, the last row of 
(9.2-10) [in reality, Eq. (9.2-9)] yields that x 2 + x 3 constitutes a state estimate 
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x e of x. Second, setting 



y = [ 0 H' 0] 



*2 

x 



x — x 7 — X 



3j 



(9.2-11) 



and adjoining this equation to (9.2-10), we can check from these two 
equations that the transfer function matrix relating u e to y is actually 
H'[sl -(F +GK')Y'G. 

We now consider very briefly the situation that occurs if the control 
arrangement of Fig. 9.2-4(e) is used. Quite clearly, for the arrangement to 
be of any use, the transfer function matrix W 4 (s) must have the poles of all 
its elements in the half-plane R e[s] < 0, to ensure stability. This will be the 
case if and only if the matrix F + K e H' + GK' has eigenvalues all with nega- 
tive real parts. Now, although the matrices F + K e H' and F + GK' both 
have eigenvalues all with negative real parts, it does not follow at all that 
F + K e H' + GK' will have the same property. For example, if F = [— 1], 
G = H == [1], and K e - K - [£], we have F + K e H' ~ F + GK' = [-£], 
whereas F + K e H' + GK' = [}]. 

Certainly, if F + K e H' + GK' does have negative real part eigenvalues, 
the scheme will be usable and have the correct steady-state behavior. In this 
case, the modes of the system can be shown to be determined by the eigen- 
values of F + GK', F + K e H' , and F + K e H' + GK', 

Calculations similar to those leading to W { (s), W 2 (s), etc., can be carried 
out when we start with a Luenberger estimator in Fig. 9.2-3. We shall assume 
that the state-space basis is so chosen that 




(9.2-12) 



where with F an n X n matrix, the matrix F e is (n — 1) x (n — 1), is the 
transpose of a companion matrix, and possesses desired eigenvalues. We 
suppose also that y is a scalar, and, for convenience, that u is a scalar. As 
usual, an overbar denotes omission of the last row of a vector or matrix. 
Finally, y = h'x = [0 0 * - • 0 l]x. 

The estimator equation is, from the last chapter, 



*e = F e x e + by + gu (9.2-13) 

Now the full state estimate x e is related to x e and y by 




(9.2-14) 



and we are required to construct k'x e . Denoting the last entry of k' by £, 
it follows that we have to construct 



k’x € + ky. 
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The transfer function relating u to this quantity is, then, 

Wl (s) = k'(sl - F e )-'g, (9.2-15) 

and the transfer function relating y to this quantity is 

w 2 (s) k\sl - FX'b. (9.2-16) 

The transfer function w 3 (s) [see Fig. 9.2-4(d)] is [1 — w^s)]' 1 , or 

w 3 (s) - 1 + k'[sl - (F e + gk'))''g. (9.2-17) 

We shall not bother with evaluating the transfer function w 4 (s) of Fig. 
9.2-4(e). This transfer function may or may not have poles in Re[s] < 0, 
depending on the precise matrices and vectors involved in arriving at it. 

The existence of [1 — w,(s)]~ 1 guarantees the existence of the general 
control scheme of Fig. 9.2-4(d), and with this existence established, it follows 
that the scheme will give the correct steady-state properties. However, it is 
necessary to check the stability properties, because the arrangement intro- 
duces (n — 1) further modes than are present when the usual Luenberger 
estimator is used. Suffice it to say, however, that similar analysis to that used 
for the case when non-Luenberger estimators are the starting point for a 
design yields a similar result: The modes of the controller-estimator scheme 
are determined by the eigenvalues of F + gk\ and the eigenvalues of F e , 
each counted twice. Of course, this latter matrix is normally chosen to have a 
desired set of eigenvalues. 

As an example, we consider a second-order, position-controller plant, 
with transfer function 1/s 2 , and with closed-loop poles at — 

Using a Luenberger estimator as the basis for the controller design, with its 
single pole at s ~ ~~ a, we shall deduce controller designs of the sort shown 
in Figs. 9.2-4(d) and 9.2-4(e). We shall examine subsequently how to choose 
the constant a. 

The appropriate F, g, etc., matrices turn out to be as listed. (They may 
be readily derived by techniques discussed earlier.) 






h' = [0 1 ] 



k* = [ — 1 —(1 + a)] and F e — —a. 

The transfer function w 2 (s) of Eq. (9.2-16), which is the transfer function of 
the feedback compensator, is 

w z (s) = -(1 + a) + - a -- = -(!.+ + «. 

s T & s ~F & 

The transfer function w ; 3 (/) of Eq. (9.2-17), which is the transfer function of 
the series compensator, is 



1 



s -f- oc 
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(a) 





Fig. 9.2-5 Controllers discussed in example. 

The control scheme is shown in Fig. 9.2-5(a). A scheme that is equivalent 
from the steady-state point of view is shown in Fig. 9.2-5(b). 

With a = 10, which would guarantee that the action of estimation was 
much more rapid than the action of the closed-loop plant, the feedback com- 
pensator transfer function becomes 

Us + 10 
s + 10 

which is evidently a lead network’’ transfer function, emphasizing high 
frequencies more than low frequencies. Evidently, the choice of larger <x will 
increase further the extent to which high frequencies will be emphasized more 
than low frequencies, and thus increase the deleterious effects of noise. An 
upper limit of 10 on a is probably appropriate. 

Figure 9.2-5(b) shows an arrangement with equivalent steady-state 
performance. The characteristic modes associated with this arrangement are 
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also dose to those of Fig. 9.2-5(a). In the second case, they are determined by 
the zeros of (^ 2 + ^ + l)(s + <x)(s + a + 1), and in the first case by the 
zeros of ( s 2 + s + l)(s + a) 2 . 

A classical design might actually have led to the scheme of Fig. 9.2-5(b), 
or, more probably, that of Fig. 9.2-5(c), which for a = 10 is almost equi- 
valent. The root locus diagram of the arrangement of Fig. 9.2-5(c) with a — 10 
is shown in Fig. 9.2-6, and it is evident what stabilizing effect the lead network 
has. 




So far in this section, we have discussed the use of controllers of dimen- 
sion n and (n ~ 1) for controlling an ^-dimensional, single-output plant. 
These controllers are constructed by cascading a linear control law with a 
state estimator. Now for special control laws, it may be that the controller 
dimension of n or (n — 1) can be reduced. We give a qualitative and non- 
rigorous argument. It is intuitively reasonable that a single integrator sub- 
system driven by the plant input and output can generate an estimate of a 
single linear functional of the plant state. Now this linear functional, perhaps 
combined with the plant output, may be sufficient to generate a desired feed- 
back signal, itself a linear functional ; if this is the case, a controller of dimen- 
sion 1 will then serve as well as a controller of dimension (n — 1). 

More generally, one can argue for multiple-input, multiple-output 
systems that certain special control laws can be implemented with controllers 
of dimension lower than usual. The characterization of such control laws 
and the design of the associated controllers is currently under investigation. 
It is interesting to speculate whether a linear optimal control problem may be 
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formulated for a prescribed plant, the solution of which will always lead to 
a low dimension controller. 

In the next section, controller simplifications are discussed for multiple- 
output, single-input systems. These simplifications are well understood, but 
do not represent the best that can be achieved in specific situations. 

Problem 9.2-1. Verify the formulas (9.2-4) and (9.2-5) for W 3 (s) and W 4 (s). 

Problem 9.2-2. Design feedback and cascade compensators for a 1/s 2 plant 
starting with a second-order estimator with estimator poles both at s = —5. 



Problem 9.2-3. Consider a plant with transfer function 1/s 3 , and with closed- 
loop poles at s — and s = —3. This corresponds to 
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[1 0 0 ] 



k' - [3 4 4]. 

Design feedback and cascade controllers based on a Luenberger estimator with 
poles at s = —5. Interpret the result using root locus ideas. 



Problem 9.2-4. Obtain state-space equations for the controller arrangement 
of Fig. 9.2-4(d), assuming the controller blocks are determined from a Luenberger 
estimator. Check that the modes of the system are determined by the eigenvalues 
of F 4 - gk', and of F e counted twice. [Hint: Obtain equations for x, x 2 , and x 3 , 
and then for i — x x — x 2 , x , and x 2 , in that order, where x denotes x less its last 
entry. 



9.3 SIMPLIFICATIONS OF THE CONTROLLER FOR 
MULTIPLE-OUTPUT, SINGLE-INPUT SYSTEMS 

Although the remarks in this section will normally have extensions to 
the most general form of multiple-output system, we restrict consideration 
here almost entirely to multiple-output plants consisting of a cascade of 
single-input, single-output plants, each of whose outputs is available. Figure 
9.3-1 shows a two-output example. The input of small plant 1 is the input of 




Fig. 9.3-1 Example of multiple-output plants considered. 
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the (large) plant, whereas a 2 vector, the entries of which are the outputs of 
small plants 1 and 2, is the (large) plant output. 

We shall indicate in this section a procedure of [5] permitting a gross 
simplification in the estimator structure when a function of the form k'x e 
is required, rather than the complete vector x e . The nature of the simplifica- 
tion is that the dimension of the whole estimator becomes one less than the 
maximum dimension of any of the small plants! Our method of attack in 
deriving this simplification will be to start with the basic Luenberger estimator 
and carry out manipulations on it. 

As we know, the problem of estimating the states of the complete plant 
immediately breaks down into independent problems requiring the estimation 
of the states of each small plant. With the notation x ie to denote an estimate 
of the state vector x t of the ith small plant, and with u i and y { denoting the 
input and output of the ith small plant, the ith estimator can be written 

x ie = F ie x ie + b i y i + gi Ui (9.3-1) 

and 






Y 

•*ie 



ly t J 



Here, of course, we are assuming that 



F. 



ie 



! b. 



X, = 



0 0 0 1 



+ gtU t 



y t = [ 0 0 



0 l]x, 



(9.3-2) 

(9.3-3) 

(9.3-4) 



and F ie has companion matrix transpose form with arbitrary eigenvalues, 
as chosen by the designer. Implicit in these equations is the assumption 
that an appropriate coordinate basis is used to write the small plant and esti- 
mator equations. 

Figure 9.3-2(a) shows a plant consisting of two small plants, with esti- 
mators of the preceding type, and with the implementation of the control 
law u — u ext + k’x e . Observe that connecting any of the u t or y. with the 
feedback quantity k'x e is a single-input, single-output linear system. Each 
of these systems, of course, has a transfer function, and the denominator of 
the transfer function connecting u { or y t to k'x e is A t (s) = det (si — F ie ). 
Accordingly, from the steady-state point of view, Fig. 9.3-2(a) is equivalent 
to Fig. 9.3-2(b), where m x (s), etc., denote polynomials. 

Now recall that the matrices F u are transposes of companion matrices, 
the characteristic polynomials of which are at the designer’s disposal. To 
achieve a single controller design with two or more small plants, we suppose 
that all those F ie of the highest dimension occurring among the complete set 
of F ie are taken as the same, and that the remaining F ie of lower dimension 
are taken to have characteristic polynomials that are factors of the charac- 
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Fig. 9.3-2 Controller structure. 
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teristic polynomials of the F le of highest possible dimension. One or more of 
the A,(j) will have maximum degree. Since the A,(s) are adjustable, we require 
that all those of maximum degree be the same polynomial, call it A(s). All 
other A,(.v) must factor A(s). This means that any transfer function /w,(.v) A : (.v) 
may be written as p,(s)/A(.s), where p t (s) = m,(s) if A,(s) has the same degree 
as A(s), and p{s) = m,(s) [A(^j/A,(.yj] otherwise— i.e. p t (s) is obtained by 
multiplying m/s ) by an extra factor. The same thing is done with the transfer 
function n/s)/A ,(s), which may be written <7 ,(s)/A(j) for some polynomial 

Figure 9.3-3(a) shows the controller structure with these special choices, 
and with the augmentation of those transfer functions m t (s)/A,(s) and 
«/s)/A/s) by surplus numerator and denominator factors where necessary. 
Clearly, the blocks with transfer functions g,(s)/A(s) and p z (s)/A(s) [or, in 
the general case, q t (s)/A(s) and p 1+1 (s)/A(s)], are immediately combinable 
by setting r 2 (s) — q t (s) + p 2 (s) [or, in the general case, r l+t (s) = q,(s) + 

A+iW]- With r,(s) = p,(s) and /- 3 (s) = q z (s), this simplification results in 
Fig. 9.3-3(b). 

We reiterate that the controllers of Fig. 9.3-2(a) and 9.3-2(b) are only 
equivalent in the steady state [since the controller of Fig. 9.3-2(b) has higher 
dimension than that of Fig. 9.3-2(a)]. The controller of Fig. 9.3-3(a) is 
obtained by specializing that of Fig. 9.3-2(b) and possibly by increasing fur- 
ther the dimension of the controller. It, too, is thus only equivalent in the 
steady state to the controller of Fig. 9.3-2(a). Finally, the controller of Fig. 

9.3- 3(b) is obtained by reducing the dimension of the controller of Fig. 

9.3- 3(a). However, it again is only equivalent to the controller of Fig. 9.3-2(a) 
(assuming appropriate selection of the matrices F ie ) in the steady state. 

Now we can make one sweeping simplification to the arrangement of 
Fig. 9.3-3(b), resulting from the fact that the denominators of all the transfer 
functions are the same. It is possible to realize the arrangement of Fig. 

9.3- 3(b), again from the steady-state point of view, with a controller of dimen- 
sion equal to the degree of A (s). One explicit way of writing down equations 
for such a controller is as follows. Each transfer function r t (s)/A(s) is repre- 
sentable as noted in Appendix B by state-space equations of the form 
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(9.3-5) 
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(9.3-6) 
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Fig. 9.3-3 Special cases of the controller of Fig. 9.3-2. 
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where v i is the input into the block of transfer function r^s)/ A(s), w. is its 
state, and z, its output. For the first block, v x coincides with the plant input 
u; for the second block, v z coincides with the output y x of small plant 1; 
for the third block, v 3 coincides with the output y 2 of small plant 2. The quan- 
tity fed back, which should be an estimate of k'x , is z t , (Of course, gener- 
alization to more than two small plants is immediate.) 

The state-space equations of the overall controller become, simply, 





y 12 

in 

) ; 3 2 



Vn 




(9.3-7) 



z = [ 0 0 



l]vv + [c5j S 2 S 3 ] v 2 • 



(9.3-8) 



Clearly, with the identification z = 2 z i9 the steady-state behavior of the 
controller defined by (9.3-5) and (9.3-6) is the same as that defined by (9.3-7) 
and (9.3-8). Again, generalization to many small plants is easy. 

Figure 9.3-4 shows the new controller, where the matrices F e and T of 
the figure have obvious interpretations. We reiterate that the dimension of 
F e is one less than the maximum dimension of the small plants. 

Although we have argued that only the steady-state performance of the 
controller is as required, it would appear that the extra modes that are intro- 
duced, above those which would be present in case true state feedback were 
employed, are all asymptotically stable. In fact, we claim that the modes of 




Fig. 9.3-4 Final controller arrangement. 
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the arrangement of Fig. 9.3-4 are determined by the eigenvalues of F + gk ' 
and the eigenvalues of F e . However, we shall not prove this result here, which 
seems difficult to establish. 

The preceding arguments will all hold if instead of using a Luenberger 
estimator for each small plant in Fig. 9.3-2(a), we use estimators whose dimen- 
sions are equal to the dimensions of the small plants. In this case, the end 
result is a controller with a dimension equal to the maximum dimension of 
any of the small plants. 

We consider now a simple example. An open-loop plant is prescribed, 
which consists of a cascade of two elements — one with transfer function 
1/4* + 1) and the other with transfer function 2/s(s + 2) (see Fig. 9.3-5). 




vt y 2 

Fig. 9.3-5 Open-loop plant. 



One state-space representation of the plant is 




y i = [i ok 

0 n r°~ 

o -2.h + Li-r 

y 2 = [2 0]x 2 . 

Suppose that it is desired to implement the control law 

u ~ u ext + [ ~ 3 —2 —4 — 2] 1 (9.3-9) 

which, it can be checked, will yield closed-loop poles at s ~ -*±yV I 
and a double pole at s — —2. A dynamic controller is to be used, the inputs 
of which are u , y 19 and y 2 . It is to be of lowest possible dimension, here evi- 
dently 1, since the maximum dimension of the two small plants is 2. 

The first task is to design estimators for the two small plants with the 
same dynamics. We shall locate the estimator poles at ^ ~ —5. With the pro- 
cedures of the previous chapter, it is straightforward to establish that the 
scheme of Fig. 9.3-6(a) constitutes an estimator for the first small plant. 
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(a) 




(b) 

Fig. 9.3-6 Estimators for small plants. 



Likewise, the scheme of Fig. 9.3-6(b) constitutes an estimator for the second 
small plant. 

When the control law (9.3-9) is implemented, we may proceed with the 
manipulation discussed earlier and shown in Figs. 9.3-2 and 9.3-3 to arrive 
at the control scheme of Fig. 9.3-7(a). Finally, the controller of Fig. 9.3-7(a) 
may be replaced by the scheme of Fig. 9.3-7(b), which is the desired con- 
troller of dimension unity. 

If desired, the calculations can be checked most easily by separately 
checking the steady-state and zero-input response properties. Figure 9.3-7(a) 
is a convenient place from which to start when checking the steady-state 
response. A succession of equivalent systems (from the steady-state point of 
view) is shown in Fig. 9.3-8. The block manipulations involved are very 
straightforward, and the details are left to the reader to verify. Now the poles 
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Fig. 9.3-7 

of the closed-loop system are at ±jVJ, with a double pole at -2, 
which means that the closed-loop system transfer function should be 

2 

R7Ti)H2F' 

It is easy to check that (s 2 + s + l)(s + 2) 2 (s + 5) = s 5 + 10s 4 + 34s 3 

+ 53s 2 + 44s f 20, and so Fig. 9.3-8(d) does, in fact, represent the desired 
ciosed-Joop system. 

The zero-input response may be examined by writing down a state- 
space equation for the scheme of Fig. 9.3-7(b). With vv denoting the state of 
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the controller, and with u cxt = 0, this equation becomes 
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for which the associated characteristic polynomial may be verified to be 
{s 2 + ^ + 1)0 + 2) 2 0 + 5). 

The extension of these results to multiple-output plants that do not con- 
sist of a cascade of single-input, single-output plants involves a good deal 
more computational effort [5]. The minimum dimension of the controller 
for a single-input, multiple-output plant is y — 1, where y is the observability 
index of the plant. The observability index is defined as follows. For a com- 
pletely observable ^-dimensional plant 



x = Fx + Gu (9.3-10) 

y = H'x, (9.3-11) 

the observability index y is the smallest integer such that [H F'H • • * (F') y ~ x H] 
has rank n. If x is an n vector and y an m vector, y evidently satisfies the 
inequality n/m <y<n — m, provided that the entries of y are indepen- 
dent. Attainment of the upper bound means that there is no reduction in 
controller dimension below the dimension of the basic Luenberger estimator. 
Attainment of the lower bound, on the other hand, generally represents a 
substantial reduction in controller complexity. 

The question of what is the minimum dimension of a controller for a 
multiple-input, multiple-output plant appears to be unanswered at the present 
time. 



Problem 9.3-1. Consider a plant consisting of a cascade of small plants with 
transfer functions 1 /(s + 1), 1/[j(j + 1)], and 1/s 2 . Obtain an optimal control law 
such that four dominant poles are located at — | ±j 1, —1, and —1. Design a con- 
troller of dimension 1 to implement this control law. 

Problem 9.3-2. Can you give a general proof that the modes of the controller 
of Fig. 9.3-4 are determined by the eigenvalues of F -f gk f and F e l 



9.4 THE SEPARATION THEOREM 



This section is confined to a brief statement of a theoretical result known 
as the “separation theorem” (see references [6] through [9]). We shall not 
attempt to prove the result, because the proof is not at all easy unless con- 
siderable background in statistics is assumed. 
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We assume we are given a linear system with additive input noise: 

x = F(t)x + G(t)u + v. (9.4-1) 

The input noise v is white, gaussian, of zero mean, and has covariance 
Q(t)S(t — t), where Q is nonnegative definite symmetric for all t. The out- 
put y of the system is given by 

y = H'{t)x + w (9.4-2) 

where w is white gaussian noise of zero mean, and has covariance R(t)S(t — t), 
where R(t) is positive definite for all t. The processes v and w are independent. 
The initial state x(/ 0 ) at time t 0 is a gaussian random variable of mean m and 
covariance P 0 , and is independent of the processes v and w. The matrices 

A A 

F, G , //, Q , and R are all assumed to have continuous elements. 

It is not possible to pose an optimal control problem requiring minimiza- 
tion of 

y={ ( x*Q(t)x + uR{t)u) dt (9.4-3) 

where Q(t) is nonnegative definite symmetric and R(t) is positive definite 
symmetric, even if one restricts the optimal u(t ) to being derived from the 
measurement X’X because the performance index V must actually be a 
random variable, taking values depending on v(-), w(*), and x(/ 0 ), which, of 
course, are random. 

To eliminate this difficulty, one can replace (9.4-3) by 

V = £| J" (x'Q(t)x + u'R(t)u) <//J (9.4-4) 

where the expectation is over x(/ 0 ) and the processes v(-) and w(-) on the 
interval [/ 0 , t J. It is understood that at time r, the measurement y( x), t 0 < t 
< t , is available, and that the optimal u(t) is to be expressed , in terms of y( t), 
t 0 <^<t. [Note: The optimal u(t) is not required to be an instantaneous 
function of y(t).] 

The solution of this problem, which has come to be known as the separa- 
tion theorem for reasons that will be obvious momentarily, is deceptively 
simple. It falls into two parts: 

1. Compute a minimum variance estimate x e (t) of x(t) at time /, using 
«(r), t 0 < t < t and y(z), t 0 < r < t. As we know, this problem has 
a solution wherein xft) is the output of a linear system excited by 
u(-) and X*)- This linear system is independent of the matrices Q(t) 
and R(t) — i.e., the same linear system generates x e (/), irrespective of 
what Q{t) and Rit) are. 

2. Compute the optimal control law u(t) ~ K'(t)x(t) y which would be 
applied if there were no noise, if x(f) were available, and if (9.4-3) 
were the performance index. Then use the control law u{t) = 
K'(t)x e (t), where x e (t) is obtained as in (1). This law is optimal for the 
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noisy problem. Notice that the calculation of K(t) is independent of 
H(t ), and the statistics of the noise. 

Evidently, the calculation of x,(t) and of the control law gain matrix 
K(t) are separate problems which can be tackled independently, Hence, the 
name “separation theorem.” Figure 9.4-1 shows the optimal controller. 




Fig. 9.4-1 Illustration of the separation theorem. 
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PART IV 



EXTENSIONS TO 
MOKE COMPLEX PROBLEMS 




CHAPTER 




OPTIMAL LINEAR REGULATORS 
WITH INPUT DISTURBANCES 



10.1 PROPORTIONAL-PLUS-INTEGRAL FEEDBACK 

The linear optimal regulator theory of previous chapters is tailored for 
applications where the effect of an initial condition or, equivalently, the effect 
of impulse-type disturbances on a plant output is to be reduced to zero in 
some optimal fashion. However, for the case when there is a sustained, slowly 
varying, input disturbance, the standard optimal regulator cannot attain and 
maintain the desired equilibrium conditions. 

A very reasonable approach to the linear regulator problem with input 
disturbances, including this latter type, would be to use a linear optimal regu- 
lator and to cancel the effect of the sustained input disturbance signal by 
the addition of a further control signal. This approach is certainly straight- 
forward, provided there is a priori knowledge of the sustained disturbance, 
for all that is required is the addition of an appropriate, slowly varying, exter- 
nal input signal to cancel the effect of the disturbance. See, e.g., schemes 
given in reference [1]. Unfortunately, in most practical applications there is 
no a priori knowledge of the disturbance. The disturbance is probably ran- 
domly varying, as, e.g., in the case of an antenna control scheme subject to 
wind forces. In this instance, if the wind force is constant over an appreciable 
interval, or slowly varying over such an interval, there will usually be a steady- 
state error in the system output. That is, the point of equilibrium for the sys- 
tem will not be the desired operating point. To remedy the situation, some 

227 




228 OPTIMAL LINEAR REGULATORS WITH INPUT DISTURBANCES 



Ch. 10 



mechanism making continual adjustments to the input is required. (In 
contrast, wind gusts are not a problem, because they consist of impulse 
disturbances with nonzero value over short intervals, and they can be 
accommodated by the standard regulator.) 

Since the disturbances often encountered are randomly varying, it might 
be thought that optimal stochastic control theory (see, e.g., reference [2]) 
would be able to take into account the randomly varying disturbances. How- 
ever, unless there is reliable a priori knowledge of the disturbance probability 
distributions, then this approach is also impractical. 

The question is therefore still before us as to how to design a linear 
optimal regulator with the desirable properties that accrue from optimality, 
and with the additional property that sustained disturbances at the input of 
the type described do not affect the equilibrium point. 

In classical control theory, the concept of proportional-plus-integral 
feedback is used to counter the effects of constant input disturbances, and 
proportional-plus-multiple-integral feedback is used to counter the effects 
of disturbances that can be expressed as a polynomial, e.g., a constant-plus- 
ramp input. Of course, with any design based on this theory, the resulting 
system does not necessarily have the desirable properties of optimal systems, 
and, unfortunately, the design procedures tend to be ad hoc, particularly 
for multiple-input systems. A review of some of the reasoning behind the 
concept of integral feedback is now given, since the concept will be applied 
in the next sections to linear optimal regulators. 

For the case when the disturbance is a constant, the inclusion of a feed- 
back path containing a single integration is sufficient for the closed-loop 
system equilibrium point to be zero, independent of the input disturbance. 
To see that this is so with a plausible but nonrigorous argument, we first 
suppose that the system output is zero. The output of the controller, in par- 
ticular the integrator output, may still be a constant value. In fact, if it has a 
value precisely that of any constant input disturbance (but opposite in sign), 
the plant input will be zero, and thus the desired operating point will be 
maintained. On the other hand, if the system output is nonzero — say, a 
nonzero constant — then the output of the controller, and thus the input to 
the plant, will be a constant signal plus a ramp signal. The ramp signal will 
tend to reduce the output of the plant to zero (assuming that the feedback is 
of the appropriate sign) and thus tend to achieve the desired zero equilibrium 
point. 

For the case when the input disturbance is a ramp function, as in the case 
of a velocity offset error, then the inclusion of a feedback path with two series 
integrations is used. For this case, the action of the controller once again is 
to provide a signal output when the system states are zero. This cancels 
the effects of the disturbances. 
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In this chapter, we construct two forms of a linear optimal regulator 
with dynamical feedback control laws. We then show that both systems have 
the various desirable properties associated with the standard regulator, and 
also the additional property that certain classes of input disturbances can 
be accommodated. (See also references [3] through [5].) 

As a means to achieve the objectives of the chapter, the next section gives 
an extension of the standard regulator theory to include the case where the 
integrand of the performance index is quadratic, not only in the system states 
and in the system input but also in the derivatives of the system input. The 
optimal control is the solution of a differential equation involving the system 
states and inputs; thus, the controller for this case is dynamic rather than 
memoryless, as for the standard optimal regulator. Two ways of implementing 
the control law are then presented, which correspond to the proportional- 
plus-integral feedback of classical control. 

The following section discusses the properties of this modified optimal 
regulator and its application to situations when there are unknown finite 
disturbances at the input to the plant to be controlled. 



10.2 THE REGULATOR PROBLEM WITH DERIVATIVE 
CONSTRAINTS 



We intimated in an earlier chapter that the standard regulator theory 
could be extended by the application of various transformations to give solu- 
tions to modified regulator problems. Using this approach, we shall solve the 
following problem. 



Modified regulator problem . Suppose we are given the linear dynamical 
system 

x = Fx + Gu x(r 0 ) given (10.2-1) 

where Fand C are constant, and the performance index 

V(x(t 0 ), u( ), t 0 ) - P ( uRu + uSu + x'Qx) dt (10.2-2) 

T t 0 

where S is positive definite symmetric, and R and Q are nonnegative 
definite symmetric. Suppose also that an initial value of the control 
u(t 0 ) is specified. Find a control w*(-) with u*(t 0 ) = u(t 0 ), which when 
applied to the system (10.2-1) minimizes (10.2-2). [Later, the case when 
u(t Q ) is to be optimized is considered.] 



As a first step in solving this problem, we define new variables 

x 
u 



r 



*1 = 



u 



u 



(10.2-3) 
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Equation (10.2-3) incorporates the transformations to be applied to the ori- 
ginal system. Now, since u(t 0 ) is specified, the system equations and perfor- 
mance index may be written in terms of the newly defined variables as follows : 

= F%x i + G l u l given (10.2-5) 

J oo 

{u\R l u l +x’ i Q 1 x t ) dt. (10.2-6) 

fo 

Figure 10.2-1 shows system (10.2-5); it consists of system (10.2-1) augmented 
by integrators at the inputs. The state vector of system (10.2-1) is augmented 
by the input vector of (10.2-1) to yield the state vector of system (10.2-5). 



f I 




Fig. 10.2-1 Augmented system. 




Fig. 10.2-2 Optimal control of augmented system. 
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u 






Fig. 10.2-3 Optimal regulator with dynamic controller. 




Fig. 10.2-4 Proportional-plus-integral state feedback. 



The standard regulator theory of the earlier chapters may now be 
applied to the augmented system (10.2-5) and quadratic performance index 
(10.2-6). Of course, various controllability and observability conditions should 
by satisfied. Assuming that they are satisfied, the form of the optimal control 
is indicated in Fig. 10.2-2. This augmented system and linear memoryless 
controller may be rearranged as in Fig. 10.2-3 to be the original system 
(10.2-1) with a linear dynamic controller. In fact, later we shall show that 
an alternative arrangement is that of Fig. 10.2-4, where the controller has 
proportional-plus-integral state-variable feedback. That these regulators 
are, in fact, the desired optimal regulators is perhaps not immediately appar- 
ent, but this should become clear as we develop the subject in more detail. 

First, we apply the standard regulator theory to minimize (10.2-6) 
subject to (10.2-5). We observe that the conditions previously imposed on 





232 OPTIMAL LINEAR REGULATORS WITH INPUT DISTURBANCES 



Ch. 10 



S, Q, and R ensure that R , and Q x satisfy the requirements of the standard 
regulator theory — viz., that R L be positive definite symmetric and Q, be 
nonnegative definite symmetric. However, two further assumptions are 
required in order that (1) the control law exist and the performance index 
be finite, and (2) the closed-loop system be asymptotically stable. 

Assumption 10.2-1. The pair [iq, G t ] is completely controllable. 

Assumption 10.2-2. The pair [F lt D{[ is completely observable for any D\ 
satisfying D l D\ = Q { . 

With the preceding assumptions, the optimal control law uf and mini- 
mum index V* associated with the minimization of (10.2-6), subject to 
(10.2-5), are given directly using the results of Chapter 3, Sec. 3.3. We have 

uf = —R x 'G\Px l (10.2-7) 

and 

^*(*,Oo), to) = x\{t 0 )P Xl {t 0 ) (10.2-8) 

where 

P = lim P(t, T) = lim P(t, T) (10.2-9) 

T — >oo J —* — CO 

with P(-,r) the solution of the Riccati differential equation 

-P = PF X +F r l P-PG l R^G\P + Q x P(T,T) = 0. (10.2-10) 

Assumption 10.2-1 ensures that P exists, and Assumption 10.2-2 ensures 
that the closed-loop system 

x x = (F x - G 1 R x l G\P)x l (10.2-11) 

is asymptotically stable. 

We now examine, in order, the various previous equations, assumptions, 
and results in the light of the definitions (10.2-3) and (10.2-4). In this way, 
we shall obtain the solution of the modified regulator problem. 

Earlier, we showed that the augmented system equation (10.2-5) follows 
from the original system equation (10.2-1), on using the definitions (10.2-3) 
and (10.2-4). The argument may be reversed without difficulty — i.e., (10.2-1) 
follows from (10.2-5), on using these same definitions. Furthermore, the 
definitions (10.2-3) and (10.2-4) imply that the two performance indices, 
(10.2-6) for the augmented system and (10.2-2) for the original system, take 
the same value. In particular, the optimal performance indices are the same 
when the initial states of the two systems are related by the first of Eqs. 
(10.2-3), and the optimal controls are related by the second of Eqs. (10.2-3). 

We now ask if Assumptions 10.2-1 and 10.2-2 can be reduced to assump- 
tions concerning the original system (10.2-1) and index (10.2-2). The answer 
is yes. In fact, Assumption 10.2-1 may be replaced by the following equivalent 
assumption. 
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Assumption 10.2-3. The pair [F, (?] is completely controllable. 

Problem 10.2-1 asks for further details on this straightforward result. 

In considering the observability Assumption 10.2-2, the arguments are 
more involved unless we make the further restriction that R be positive defi- 
nite rather than simply nonnegative definite. For the simpler case, Assump- 
tion 10.2-2 may be replaced by Assumption 10.2-4. 

Assumption 10.2-4. The matrix R is positive definite, and the pair [F, 2) n ] is 

completely observable, where D u is any matrix such that D n D\ l ~ Q. 

Problem 10.2-2 asks for details on this result. Notice that we have only 
claimed that Assumption 10.2-4 implies Assumption 10.2-2, and not the 
converse. The condition that R be positive definite may be relaxed with cau- 
tion (see Problem 10.2-3). 

Now that we have interpreted the equations and assumptions associated 
with the augmented system in terms of quantities associated with the original 
system, we turn to the interpretation of the results of the minimization 
problem for the augmented system, in order to give a solution to the minimiza- 
tion problem for the original system. As remarked earlier, the optimal con- 
trol w* for the modified regulator problem satisfies w* — u x with u*(t 0 ) equal 
to the specified u(t 0 ). The minimum index, being the same for both minimiza- 
tion problems, is K*(x(/ 0 ), w(/ 0 ), f 0 ) — ^*(x,(/ 0 ), t Q ). The optimal control 
w* and the minimum index V* can now be expressed in terms of the modified 
regulator problem parameters as follows. 

Partitioning P as 




the optimal control u* is given from 

u* = uf w*(/ 0 ) — M/o) specified 
= ~R^G\Px 1 




= —S~ l P 2l x - S~ l P 12 u** 

That is, the optimal control u*(*) is given from 

w* = K[x + K 2 u* u*(r 0 ) = u{t 0 ) specified (10.2-12) 

where 

K[ = ~S^P 2l] K' = -S-'P 21 . (10.2-13) 

See Fig. 10.2-3 for a diagram of a controller that achieves this optimal con- 
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trol law. The minimum performance index associated with this control law 
is given as 

^OOo), «Oo), to) = ?o) 



= C^'Oo) 



At o)] 



~P u 


pt A 

F 21 


Ao) 


At 


P 

1 2 2-J 


A‘o)_ 



X ^So)P\ l^(^o) t - 2w (/q)^ l-*Vo) ’ tl (/ 0 )/*22^(^o)* 



(10.2-14) 

It is interesting to note that for the case when w(r 0 ) is zero, the minimum index 
reduces to 



V*(x(t 0 ), u(t 0 ) = 0, /„) = r'(/ 0 )P„4). (10.2-15) 

Until now, we have assumed that u(t 0 ) is specified. However, it is clear 
that if w(r 0 ) is chosen to minimize (10.2-14), then it would have the value 

w*('o) = -NlhXto) = -{K'J-'K'.xU 0 ). (10.2-16) 

For this case, 

V*(x(t 0 ), t 0 ) = *'(/ 0 )LP u - PhPiiP 2 Mt 0 )- (10.2-17) 

These results are perhaps only of academic interest, because in practice there 
would usually be no more direct control of the initial states u(t 0 ) of the con- 
troller than there would be of the initial states x(t 0 ) of the plant. 

The optimal control law and minimum index are expressed in terms 
of P n , P 2U and P 22 . These matrices are simply the limiting values as t 
approaches minus infinity of P lu P 2i , and P 22 where [see (10.2-9) and 
( 10 . 2 - 10 )] 

=PuF + F'P u -P' 21 S~'P U +Q P x j (T) = 0 

-^21 = P 2l F + G'P n -P 22 S'P 2i Ph(T) = 0 

-P 22 =P 2l G + G f P' 2l -P 22 s-'P 22 +R P 22 (T) = 0. 

(10.2-18) 

The results developed in the preceding paragraphs constitute a solution 
to the regulator problem with derivative constraints. However, there is an 
alternative formulation of the optimal control law w*, which we now derive. 

The input to system (10.2-1) may be expressed in terms of the state x 
and its derivative x from (10.2-1) as follows: 

u = (G'G)-'G'(x - Fx). (10.2-19) 

For the preceding equation to be meaningful, we require that G’G be positive 
definite or, equivalently, that G have rank equal to the number of system 
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inputs. For a completely controllable single-input system, G'G will be a posi- 
tive constant and thus invertible. For a multiple-input system, the physical 
interpretation of this condition on G is as follows. If the rank of G were not 
equal to the number of system inputs or number of columns of G, it would 
be possible to delete one or possibly more inputs and, by adjustment of the 
remaining inputs, to leave the system trajectories unaltered. In other words, 
the system is in a sense over controlled. Clearly, it is always possible to 
arrange the system such that this condition is avoided. 

Assuming then that G'G is positive definite, and thus that (10.2-19) 
holds, the differential equation (10.2-12) may be written 

u* = K' 3 x + K 4 x u*(t 0 ) ~ u(t 0 ) specified (10.2-20) 

where 

K' 3 - K\{G f G)~ l G' K 4 K\ - A 3 F. (10.2-21) 

Integrating (10.2-20) gives 

«*(0 = K’s x(t) + f K f 4 x(r) dr + u(t Q ) ~ A' 3 x(r 0 ). (10.2-22) 

From this equation it is clear that the optimal control w*(*) can be realized 
by a proportional-plus-integral state feedback control law (see Fig. 10.2-4). 

In practice, the initial state of the integrator in Fig. 10.2-4 will not be 
directly controlled any more than is the initial state of the plant x(t 0 ). For 
the case in which we can choose an initial state for the integrator so that the 
performance index is minimized, clearly, from (10.2-16) and (10.2-22), this 
initial state would be — K' 3 ]x{t Q ). 

We have the results that the optimal controller for our modified regulator 
problem may be either of the form shown in Fig. 10.2-3 or of the form shown 
in Fig. 10.2-4. The controllers for each case will be different but the resulting 
control w*( • ) will be the same. The equation of the closed-loop system in either 
case will be the same as that for the optimal augmented system of Fig. 10.2-2 
— viz., (10.2-11) or, equivalently, 



~ X 




~ F G~ 


~ X " 




x(t 0 ) ~ 


u* 




K\ K\_ 









specified. 



The various preceding results are now summarized. 



(10.2-23) 



The solution to the regulator problem with derivative constraints . For the 
completely controllable system (10.2-1) and performance index (10.2-2), 
the optimal control u*(«) is given from 

u* = K\x + A' 2 w* «*(f 0 ) — - u{t 0 ) specified ■ (10.2-12) 

or, equivalently, for the case when (G'G) is positive definite, from 

m* = K f 3 x + K' 4 x i/*(/ 0 ) = u(t 0 ) specified (10.2-20) 
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where K l9 K 2 , K 3 , and K \ are given from (10.2-13), (10.2-18), and 
(10.2-21). (See also Figs. 10.2-3 and 10.2-4). The minimum index F*(x(? 0 ), 
u(t o), / 0 ) is given as 



y*(x(t o), w(f 0 ), / 0 ) = x'('o)Ai*Oo) + 2i/(t 0 )P 21 x(/ 0 ) 

T W (/o)^22^(^o)‘ 

A sufficient condition for the closed-loop system 

specified 



~ X 


_ 


~ F 


G~ 




X 




1 — 
o 


u* 




K\ 


K’i_ 




u * 




_w(/ 0 L 



(10.2-14) 

(10.2-24) 



to be asymptotically stable is Assumption 10.2-4. For the case when 
u(t 0 ) =-- 0, the minimum index is given as in (10.2-15), and for the case 
when u(t Q ) may be chosen so that the index is minimized, then the optimal 
u(t 0 ) is given in (10.2-16) and the associated index in (10.2-17). 



There is one immediate extension of the preceding results. By consider- 
ing the augmented system (10.2-5) as the ‘‘original system” and applying the 
same procedure already outlined for this sytem, then we may interpret the 
results as constituting the solution of the regulator problem with second- 
order derivative constraints. The controllers for this case will contain a 
further stage of integration than those already discussed. The applications of 
this extension to the theory will be discussed in the next section, whereas the 
details of the extension process itself are left to the student (see Problem 
10.2-4). Extensions are also possible to the case when the original system is 
augmented by an arbitrary dynamical system. These extensions have applica- 
tion (see references [6] through [8]) to the construction of dynamic compen- 
sators for systems with incomplete state measurements. Problem 10.2-5 asks 
for details to be worked out for a particular example. 



Problem 10.2-1. Using the notation of the section, show that [Fj, G,] com- 
pletely controllable is equivalent to the condition [F, G] completely controllable. 
Use the controllability rank condition (see also Appendix B). 

Problem 10.2-2. Using the notation of the section, show that Assumption 
10.2-4 implies Assumption 10.2-2. 

Problem 10.2-3. Suppose you are given the completely controllable system 
x = Fx + Gu and the performance index 

Foo 

V = (uSu -f x'Qx) dt. 

J 0 

Suppose also that D is a matrix satisfying DD' = £), and that [F, D] is completely 
observable. Show that the pair 



"F 


G~ 




^ D 0~ 


_0 


0_ 




_0 0_ 
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fails to be completely observable if and only if there exist x 0 and « 0 , not both zero, 
such that Fx 0 -f <ju 0 = 0 and D'j c 0 = 0. Give a simpler statement of this condition 
for the case when Fis nonsingular. Show also that, if the condition holds, the closed- 
loop system is not asymptotically stable. 



Problem 10.2-4. For the completely controllable system x = Fx 4- Gu and 
performance index 

Foo 

V = J (ii Sii + x'Qx) dt , 

find the optimal control law that minimizes the index V. Determine if the control 
law may be realized using proportional-plus-integral-plus-double-integral state- 
variable feedback. 



Problem 10.2-5. For the system 




0 1 
.0 0j 



*i 

-* 2 - 



+ 




U 



y = Xi - x 29 

the control that minimizes the index 



is 



V = J (ii 2 -h x\) dt 



u = —X! — 2x 2 — 2m. 



Show how this control law may be realized using output, rather than state-variable 
feedback. Show that the feedback compensator has a structure similar to a 
Luenburger estimator with state-variable feedback. (Hint: First express u as 

« = -a 0 w — Pa y - 

See reference [6]. References [7] and [8] give generalizations of this procedure.) 



10.3 APPLICATION AND PROPERTIES OF OPTIMAL 

REGULATORS WHICH INCLUDE INTEGRAL FEEDBACK 



Using the theory of the previous section, linear optimal regulators that 
include single- or multiple-integral feedback may be designed in a straight- 
forward manner. We now consider the properties of these systems, and, in 
particular, their ability to accommodate input disturbances. 

Consider the completely controllable system with state equation 

* = Fx + G x u + G 2 w x(t 0 ) = 0, u(t Q ) - 0 (10.3-1) 

where w is a constant disturbance vector. Note that for simplicity we have 
chosen zero initial conditions. (Nonzero initial conditions can readily be 
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incorporated, if required.) Now, to achieve our objective — that the state x 
and its derivative x be zero as time approaches infinity — we clearly require 
the following relationship to hold: 

G,u(oo) = — G 2 m>(oo). (10.3-2) 

This condition can be satisfied if and only if it is possible to choose a matrix 
M such that G 2 = G X M^ because then w(oo) can be chosen as u{oo) = 
— Mw( oo). That is, we require the following assumption. 

Assumption 10.3-1. The range space of G 2 is contained in the range space of 
Ci. 

With Assumption 30.3-1 holding, we may write the system equations 
(10.3-1) as 

x = Fx + G(u + w) (10.3-3) 

where w = Mw, G = G\, and [F, G] is completely controllable. 

For the system (10.3-3), a performance index with derivative constraints 
corresponding to that of the previous section, Eq. (10.2-2), is 

V(x(t 0 ) 9 «(•)> / 0 ) = f [(u + w)'R(u + w)+ u'Su + x'Qx ] dt (10.3-4) 

j to 

where S is positive definite symmetric, and R and Q are nonnegative definite 
symmetric. Applying the results of the previous section, we have that the 
optimal control w*(*) which minimizes the preceding index is given from either 
the differential equation 

w* - K\x + K f z (u* + w) u*(t 0 ) - 0 (10.3-5) 

or from the alternative equation 

ii* — K$x T K^x u*(t Q ) = 0. (10.3-6) 

Note that we have used the fact that (d/dt)(w + u) = The matrices 
K 2 , K 3 , and K A are given from 

K\ = -S~'P 2l K\ = —S' X P 22 K’ 3 = K 2 (G'Gy i G' K\ = K\ + K\F. 

(10.3-7) 

The matrices P 22 and P 2i (and also F n ) are given as the limiting values as 
t approaches minus infinity of P 22 and P 2l (and P n ). Here, P n , P 12 and P 22 
are given by 

-P ll =P u F + F f P u -P f 2l S' 1 P 21 +Q P lt (T) = 0 

-Pn = Pz\F + G'P U ~P 22 S' 1 P 2l Pu(T) = 0 (10.3-8) 

-P 22 = P 2 \G + G'P' 21 - P 22 S' 1 P 22 + R P 21 (T) = 0. 

The minimum index is also given for this minimization problem as 

V*(w, t Q ) = w'P 22 w. (10.3-9) 
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Furthermore, if the pair 

f ci r D u 0 “ 

-0 oj’ Lo d 22 _ 

is completely observable for any D u such that D, , D \ , = Q, and any D 12 such 
that D 22 D' 22 — R, then the closed-loop system 

* 1 m r-F Gir X n r jt(/ 0 ) iron 

u -f- vv_ _u*_ _K\ K' 2 _ u* w_ + w_ _w_ 

(10.3-10) 

is asymptotically stable. Useful sufficient conditions are that [F, Z) n ] be 
completely observable and R be positive definite. 

The important point to observe from the asymptotically stable equation 
(10.3-10) is that the equilibrium condition is x = 0, x = 0 and u* = —w. 
as desired. Figure 10.3-l(a) shows the scheme for realizing the control law as 
expressed in (10.3-5), whereas Fig. 10.3-1 (b) gives the scheme for the same 





Fig. 10.3-1 Optimal regulator configurations. 
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control but expressed in the alternative form (10.3-6). Note that the second 
scheme is literally proportional-plus-integral (state) feedback, whereas the 
first scheme consists of integral feedback of the states and plant input. Clearly, 
if the input is not available, the first scheme cannot be used. 

We stress again that through use of either of the forms of the optimal 
linear regulator with input derivative constraints shown in Fig. 10.3-1, the 
states of the original system (10.3-3) will approach an equilibrium value 
of zero asymptotically irrespective of the value of the constant input disturb- 
ance. The control will approach the negative value of the input disturbance 
asymptotically, thus cancelling its effect. 

Now that a construction procedure for linear optimal regulators that 
include integral feedback has been given, and it has been established that 
constant input disturbances are cancelled by the controller output as time 
approaches infinity, we ask if such regulators have the same desirable prop- 
erties as the standard optimal regulator. The answer is yes. Some of the prop- 
erties can be demonstrated immediately, because the modified regulator is 
simply a standard regulator in disguise. Clearly, by replacing g, S , and R 
with Qe 2a \ Se 2at and Re 2aLt , respectively, or, equivalently, by replacing 

F G~ 

_0 0 _ 

by 

~F + a I G' 

_0 a/_ 

in the derivations, an optimal system can be obtained with a degree of stability 
of at least a (see Problem 10.3-1). Again, it is clear that the sensitivity prop- 
erties of the standard regulator carry over almost immediately into the 
modified regulator (see Problem 10.3-2). The question as to whether either 
of the modified linear optimal regulators can tolerate nonlinearities at the 
plant input will now be considered. 

Let us suppose that a nonlinearity is interposed at the plant input, as in 
Fig. 10.3-2(a) or 10.3-2(b), with property that it changes the nominal plant 
input a to 0(a), where 0(*) is acontinuous function ofcr, satisfying the inequal- 
ity constraint (i + e)o'o < a'0(a) for arbitrary positive e — a constraint 
familiar from earlier discussions. [For the case of a scalar a, this requires the 
graph of 0(a) to lie between straight lines of slope ^ and oo passing through 
the origin. See Fig. 10.3-3.] 

To study the stability properties of the closed-loop plant and controller 
scheme with the newly introduced nonlinearity, we study first the scheme of 
Fig. 10.3-2(c), known from standard regulator theory to be asymptotically 
stable. Now, the system of Fig. 10. 3- 2(c) may be arranged as a linear subsys- 
tem of transfer function matrix 





Fig. 10.3-2 Various forms for the 



one optimal regulator. 
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Fig. 10.3-3 Sector nonlinearities. 




Fig. 10.3-4 Regulator with nonlinearities. 



W(s) = s 'K\(sI- F) l G +s~ l K f 2 (10.3-11) 

with feedback nonlinearities $(•), as indicated in Fig. 10.3-4. We claim now 
that the systems of Figs. 10.3-2(a) and 10.3-2(b) (which have the same 
form as the system of Fig. 10.3-4) are, in fact, identical to the system in Fig. 
10.3-4, with the transfer function matrix of the linear part being tV(s), as in 
(10.3-11), in both cases. Consider first the arrangement of Fig. 10.3-2(a); 
the transfer function matrix of the linear subsystem is 

K[(sl- Fy'Gs 1 + K' 2 s~ l = W(s). 

For the system of Fig. 10.3-2(b), the linear subsystem transfer function matrix 
is 

(s~ x K\ -r K' 3 ){sl ~ F)~ l G - (s~ l K[ - s'K’ 3 F + K' 3 )(sl - F) X G 

= [s- x K\ + 5 - 1 K' 3 (sI ~ F)](sl - Fy l G 
- s~ x K' 3 G + s ' 1 K\(sI — F)~ 1 G 
= W(s). 

In the preceding development, we have used the definitions for K 3 and 
given in the previous section, Eq. (10.2-21). 

We conclude that since all three systems of Fig. 10.3-2 can be arranged 
in the form of Fig. 10.3-4, with W(s) given by (10.3-1 1), and since the system 
of Fig. 10.3-21 c) is known to be asymptotically stable, using standard regulator 
theory, then the optimal regulators of Figs. 10.3-2(a) and 10.3-2(b) are also 
asymptotically stable. Note, in particular, that for the linear optimal regulator 
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with proportional-plus-integral feedback, the nonlinearities are in the posi- 
tion of the plant input transducers. (A generalization of this result is requested 
in Problem 10.3-3.) 

The preceding results also indicate that the gain margin and phase margin 
properties of single-input linear regulators, and their ability to tolerate time 
delays within the closed loop and still remain asymptotically stable, carry 
over to the modified regulators of Fig. 10.3-2. 

To conclude the section, we discuss aspects of a second-order example. 
Consider the case when 





'0 


r 




ro~ 




'1 0' 


F — 
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0 
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G = 
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e = 





5=1 and R = 0. To apply regulator theory to the system augmented by 
integrators at the inputs, we require that certain controllability and observ- 
ability conditions be satisfied. Specifically, we require that [F, G] be com- 
pletely controllable and that the pair 



~F 


G~ 
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D 0"| 
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> i 
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1 

o 

o 

1 



be completely observable for any D such that DD' = Q. Since the matrices 
are time invariant, these conditions are perhaps most readily checked using 
the rank conditions, which are necessary and sufficient conditions for com- 
plete controllability and observability of time-invariant systems. 

For [F, G] to be completely controllable, the rank condition is 



Rank (G FG F 2 G F^G) = n (10.3-12) 

where n is the order of the system. For [ F , D'] to be completely observable, 
the rank condition is 



Rank (D F'D (F') 2 D - - - (F r ) n ~ l D) = n. 

Applying these conditions to our example, we have 



(10.3-13) 



Rank 



Rank (G FG) = Rank 



0 1 
1 0 



= 2 



r & o' 

0 0 



n F' 0“ 
G' 0 



D' O' 
0 0 



= Rank 



= Rank 



= 3. 



D' F'D ' 
u 0 G'D'j 
10 0 0 
0 110 
0 0 0 1 



L 



We conclude that [F, G] is completely controllable and the pair 
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1 



is completely observable, as required. 

To complete the solution of the problem, we may apply standard regu- 
lator theory in a straightforward manner to the augmented system, which is 

x { = F x x x + G x u A 

where x, — [x' u]' and u x = w, with 
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The performance index is 

1*00 

V = {x\Q l x l +u\R i u l )dt 

J to 

where 



fi. = 







_ 1 
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O' 


~Q 
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_0 


R 














_0 
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0_ 



R, = 5= 1. 



Hence, we find P = lim P(t, T), where P(*, T) is the solution of 



f-»-oo 



- P = PF, + FiP - PG X R~ X X G\P + Q F(F, F) - 0. 
The solution is 



Pll 


: P' 

j r 21 


p 2l 


! p' 

\ r 22 



2 2 
2 3 



1 

2 



1 2 

The optimal control to the augmented system is 

u* = -Rx l G\Px x = -[1 2 2]x, 

or 



w* = — [1 2]x — 2u , 

but jc = Fx + Gw, and thus, 

u = (G'G)~ x G f (x - Fx) = [0 1]jc, 

Combining the preceding two equations gives 

u* = -[1 2]x - [0 2]x. 

The response of the system jc = Fx + G(w + w) for a constant w with this 
control law (viz., w* = — [0 2]x — J f 0 [1 2]x dt) is plotted in Fig. 10.3-5(a). 
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The control u* is plotted in Fig. 10.3-5(b). Corresponding quantities when a 
control law is derived from a standard regulator problem with 

. n 01 



R == 1 and S = 0, are given for comparison purposes in the same figures, 
using dotted lines. It is clear that the regulator with integral feedback can 

accommodate constant offset errors w, whereas the standard regulator can- 
not. 




regulator with integral feedback — 
standard regulator 




Fig. 10.3-5 Optimal control and response of regulators. 

Problem 10.3-1 . Repeat the derivation of the optimal linear regulator, which 
includes integral feedback for the case where the specifications require a guaranteed 
degree of stability of at least a. 

Problem 10.3-2. Apply the sensitivity results of Chapter 7 to the optimal 
linear regulator which includes integral feedback, and thereby determine its sen- 
sitivity properties. 
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Problem 10.3-3. Consider the system x = Fx -f Gu , where the optimal 
control u satisfies the differential equation (u (p) ~ d p uldt p ) 

u {p) =*£K t u<» + K p x. 
o 

An ancillary property of optimality is that when u (p) is replaced by 0(« (p) ), where 
( p represents nonlinearities as indicated in the text, the closed-loop system is asymp- 
totically stable. Applying the relationship 

u< n = (G / G)~ 1 G'(x u+1) - Fx u >) 9 

we may rewrite the preceding control law as 

u {p) = *£ Ai(G'G)- 1 G'x« +l > - P t! Ki{G'G)~ x G'Fx U) + K p x . 
o o 

Using this law, show that the closed-loop system is asymptotically stable, with u 
replaced by <f>(u) [rather than u (p) replaced by (f)(u ip) )]. (Hint: This is a generaliza- 
tion of a result developed in the section. To solve the problem, equate transfer 
functions, as is done in the text for the case p = 1.) 
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CHAPTER 



11 



TRACKING SYSTEMS 



11.1 THE PROBLEM OF ACHIEVING A DESIRED 
TRAJECTORY 



In previous chapters, the regulator problem — viz., the problem of return- 
ing a system to its zero state in some optimal fashion — is considered. This 
problem is, in fact, a special case of a wider class of problems where it is 
required that the outputs of a system follow or track a desired trajectory in 
some optimal sense. For the regulator, the desired trajectory is, of course, 
simply the zero state. In this chapter, we apply regulator theory and give 
extensions to solve the wider class of control problems that involves achiev- 
ing a desired trajectory. 

It is convenient to refer to these problems by one of three technical 
terms, the particular term used depending on the nature of the desired trajec- 
tory. If the plant outputs are to follow a class of desired trajectories, e.g., 
all polynomials up to a certain order, the problem is referred to as a servo 
(servomechanism) problem ; if the desired trajectory is a particular prescribed 
function of time, the problem is called a tracking problem . When the outputs 
of the plant are to follow the response of another plant (or model) to either 
a specific command input or class of command inputs, the problem is referred 
to as the model-following problem . 

The remainder of this section is devoted to a discussion of considerations 
common to all three of these problems, with particular attention being given 
to the selection of a performance index. 

We recall that in selecting a performance index for a regulator, cost 
terms are constructed for the control energy and the energy associated with 
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the states. More specifically, for the linear system 

x — Fx + Gu x(t Q ) given (11.1-1) 

the principal performance index adopted throughout the book is the quad- 
ratic index 

V(x(t Q ), w(«), 7 1 ) = [ 0 u'Ru + x'Qx ) dt (11 .1-2) 

** to 

where R is some positive definite matrix and Q is some nonnegative definite 
matrix (the matrices being of appropriate dimensions). The quadratic nature 
of the cost terms ensures that the optimal control law is linear, and the con- 
straints on the matrices Q and R ensure that the control law leads to a finite 
control. 

When attempting to control the system (11.1-1) such that its output 
>>(•) given by 

y = H'x (11.1-3) 

tracks a desired trajectory y(*)> there clearly should be a cost term in the per- 
formance irdex involving the error (y — y). A performance index that comes 
to mind immediately as a natural extension of the index (1 1.1-2) is the follow- 
ing: 

V(x(t Q \ «(•), T) - f [u'Ru + (y - y)'Q(y - y)] dt (11.1-4) 

J to 

where Q aid R are nonnegative definite and positive definite matrices, 
respectively. Once again, we have quadratic terms that, as the next sections 
show, give rise to linear control laws. 

Another objective, apart from minimizing the error (y — y) and the 
control w, may be to achieve a smooth system output y. It may be that the 
error (y ~ y) is not critical at every instant in time, but a smooth response is 
critical. One possible method that may be used to achieve the objective of 
smoothness is to generalize the index (11.1-4) as follows: 

V(x(i 0 ), w(-), T) = f [u'Ru + x'Q + O — y)’Q z (y — y)] dt 

J to 

(11.1-5) 

where Q x ar d Q 2 are nonnegative definite symmetric matrices. 

The term x'Q^x tends to constrain the system states to be small, and 
thereby to eicourage a smooth response. (A response that is not smooth 
must contain significant amounts of the output derivatives; thus, at least 
some of the system states must accordingly be large.) Unfortunately, without 
a restriction on Q u the two objectives represented by the second and third 
terms in (11,1-5) may be conflicting, for the following reason: The term 
x Q x x is to encourage the entries of x to be small, and, in particular, the 
entries of y -- H'x to be small, whereas the term (y — y)'Q z (y — y) is to 
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encourage the e ntries of y = H'x to be close to those of y, which, in general, 
will not be small. 

We now claim that this difficulty can be overcome by restricting Q x 
to being of the form 

<2i = [/ - H(H'H)-'H']'Q 3 [I - H(H'H)"H'] (11.1-6) 

where Q } is an arbitrary nonnegative definite symmetric matrix. [It is implicitly 
assumed here that exists. This restriction is equivalent to the very 

reasonable assumption arising from practical considerations, that the system 
outputs are linearly independent.] 

To establish the preceding claim, we first write x as the sum of two ortho- 
gonal components x l and x 2 , such that x { is in the range space of H\ That is, 



we write 




* = *! + * 2 


(11.1-7) 


where 




HI = x 1 and H'x 2 ~ 0 


(11.1-8) 


for some vector /. These equations imply that 




y = H'x - H'x t , 


(11.1-9) 



which, in turn, implies that constraining y to follow y constrains x l9 via 
(11.1-9), whereas no constraint on x 2 is implied. We conclude that any pair 
of objectives represented by cost terms x 2 Q 3 x 2 and ( y — y)'Q 2 (y ~ y) would 
not be in conflict for any Q 3 , whereas, as we have already seen, a pair of 
objectives represented by cost terms x'Qx and (y — y)' Q 2 (y — - y) may be in 
conflict. 

But now it is easy to check that the vector / as defined is 

( 11 . 1 - 10 ) 

and, accordingly, 

x 2 = x — ///=[/- H{H'Hy i H']x. ( 11 . 1 - 11 ) 

Therefore, Eq. (11.1-6) implies that x f Q A x = x' 2 Q 3 x 2 . Thus, since we have 
shown that the terms x’ 2 Q 3 x 2 and (y — y)'Q 2 (y ~ y) are not conflicting in the 
sense implied by the previous discussion, and since *' 263*2 ~ x'Q { x for the 
case when Q x is defined as in (11.1-6), then the performance index (11.1-5) 
with Q { given by (1 1.1-6) and Q 3 arbitrary has three cost terms that are not 
in direct conflict with one another. The first term is the control cost, the 
second is a “smoothness” cost, and the third is an error cost. 

We now show that the index (11.1-5) with Q x defined by (11.1-6) may be 
written in an interesting form that will prove convenient to use in the next 
sections. More precisely, we shall show that the index (11.1-5) may be written 

K(x(f 0 ), »(•), T) = f [u'Ru +(x- x)'Q(x - x)] dt (11.1-12) 
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where 

e = Cl +HQ z H' (11.1-13) 

5c - //[/f'//]- 1 y. (11.1-14) 

The irterpretation of the terms in the preceding index is straightforward 
enough. It appears that the cost terms of the index (11.1-5) involving the 
state and the error between the system output and the desired output are 
replaced by a single term involving the error between the state and some spe- 
cial state trajectory x, related to the desired output trajectory y. We have 
from (11.1*14) that H’x = y, and thus, if by some means x were to become 
the state trajectory of (11.1-1), then the system output given by (11.1-3) 
would, in fact, be the desired output trajectory y. What characterizes the 
trajectory x is the important property, to be proved, that its component x 2 
in the null space of H f is zero. In other words, if the system output vector y 
were augmented by another vector output y = Hx , with each entry of y 
linearly independent of the entries of y and with the combined dimension of 
y and y equal to the state-vector dimension, then with 5c as the state trajectory 
this output y would be zero. The specified properties of x suggest that it 
be referred to as the desired state trajectory . 

We have yet to prove that x 2i the component of x in the null space of 
H\ is zero, and also that the indices (11.1-5) and (1 1.1-12) are, in fact, iden- 
tical. Using reasoning similar to that used in deriving (1 1.1-1 1), we have that 

jc 2 — [/ — HiH'Hy'H'lx 

= 0 . 

This result yields immediately that x'Q { x = 0 and, in fact, Q^x = 0, where 
Q x is defined as in (11.1-6). It then follows that 

(x - x )'Q(x - x) = x'Q x x + (x - x)HQ 2 H\x - x) 

= x'Q t x + (y — y)'Q 2 (y — y) 

where Q is defined as in (11.1-13). Our claim is established. 

For any particular application of the performance index just developed, 
selections have to be made of the matrices Q 29 g 3 , and 7?. Also, a selection 
may have to be made of the terminal time T. It may be necessary to try a 
range of values of these quantities and to select the particular one which is 
most appropriate for any given situation. A case of particular interest is the 
limiting case as the terminal time T approaches infinity. For this case, when 
all the matrices are time invariant, part, if not all, of the optimal controller 
becomes time invariant. However, difficulties may arise for this case because 
it may not be possible for the plant to track the desired trajectories so that 
the error approaches zero as time becomes infinite with a finite control law. 
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Moreover, even if it is possible to track in this sense using a finite control 
law, unless the control approaches zero as time approaches infinity, other 
difficulties arise due to the fact that the performance index would be infinite 
for all controls and therefore attempts at its minimization would be mean- 
ingless. 

The next section considers finite terminal-time servo-, tracking-, and 
model-following problems. The final section considers the limiting situation 
as the integration interval becomes infinite. Of particular interest in this 
section are the eases where the desired trajectory is either a step, ramp, or 
parabolic function. Key references for the material that follows are [i] 
through [3]. 

11.2 FINITE-TIME RESULTS 



The servo problem. As stated in the previous section, the servo problem 
is the task of controlling a system so that the system output follows a pre- 
scribed signal, where all that is known about the signal is that it belongs to 
a known class of signals. We consider a particular specialization. 



Optimal servv problem. Suppose we are given the w-dimensional linear 
system having state equations 

x — Fx + Gu x(/ 0 ) given (11.2-1) 

y ~ H'x (11.2-2) 

where the m entries of y are linearly independent or, equivalently, the 
matrix H has rank m. Suppose we are also given an m-vector incoming 
signal y, which is the output of the known /^-dimensional linear system 

z=-Az (11.2-3) 

y = C‘z (11.2-4) 

for some initial state z(t 0 ). Without loss of generality, the pair [A, C] 
is completely observable. The optimal servo problem is to find the opti- 
mal control u* for the system (11.2-1), such that the output y tracks the 
incoming signa< y, minimizing the index 

r(x(to)M-),T)= f t {u'Ru 

J to 

+ *'[/ - H(H'H)- l H’]'Q i [I - H(H'H)-'H']x 

+ (y-yYQz(y~y)}dt (11.2-5) 

where Q x and Q 2 are nonnegative definite symmetric matrices, and R 
is positive definite symmetric. (As usual, all the various matrices are 
assumed to have continuous entries.) 
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Observe that we are requiring that our desired trajectory y be a solution 
to a linear differential equation. This, of course, rules out trajectories y 
which have discontinuities for t > / 0 . We also note that the special case 
when C is a vector and A is given by 



0 1 • . . 0 “ 
•01 






leads to the class of y consisting of all polynomials of degree (p — 1). 

Proceeding with the solution of the servo problem, we note an equivalent 
expression for the index (11.2-5), derived in the previous section — namely, 



V(x(t 0 ), «(.), T) = f [uRu + (x- x)'Q(x - x)) dt (11.2-6) 

J to 

where 



Q == [/- H(H'H)- X H']'Q X [I- H(H'H)' l H'] + HQ 2 H‘ (11.2-7) 
and x, the desired state trajectory, is 

x = H(H'H)^y. (11.2-8) 

Throughout the book, various minimization problems are solved by 
first applying a transformation to convert the minimization problem to a 
standard regulator problem. The standard regulator results are then inter- 
preted, using the transformations to give a solution to the original mini- 
mization problem. This will also be our method of attack here. To convert 
the preceding servo problem to a regulator problem, we require the following 
assumption. 



Assumption 11.2-1. The state zis directly measurable. (Later, we shall discuss 
the elimination of this assumption.) 



We now define a new variable 



and new matrices 




Z' 

F = 


~F 


Cf 


A 


~G~ 


0 




G = 


_0_ 





Q -QH(H'H)- l C’ 

- C(H’H) ~ X H'Q C(H'H) - 1 //' QH(H'H) ~ 1 C 



(11.2-9) 



( 11 . 2 - 10 ) 
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These variables and matrices are so constructed that when applied to the 
problem of minimizing (11.2-6) with the relationships (11.2-1) and (11.2-3) 
holding, we have the standard regulator problem requiring minimization of 
the quadratic index 



V(x(t a ), «(•), T) = I* (u'Ru + x’Qx) dt (11.2-11) 

J fo 

with the relationship 

% = Fx + Gu x(t 0 ) given (11.2-12) 

holding. This result may be checked by the reader as an exercise. 

Applying the regulator theory of Chapter 3, Sec. 3.3, to the minimiza- 
tion problem, (11.2-11) and (11.2-12), gives immediately that the optimal 
control u* is 

u * - -R-'&Px (11.2-13) 

where P(-) is the solution of the Riccati equation 

♦ 

A A ^ -A A A A A A A 

-P == PF + F P - PGR-'G'P + Q P(T ) = 0. (11 .2-14) 

The minimum index is 

v*(£(t 0 ), T) - x'(t 0 )p(t 0 )x(t 0 ). (1 1.2-15) 



We now interpret these results in terms of the variables and matrices 
of the original problem, using the definitions (11.2-9) and (1 1.2-10). First, we 

A 

partition P as 




(11.2-16) 



where P is an n X n matrix. Substituting (1 1.2-10) and (11.2-16) into (11.2-13) 
gives the optimal control u* as 



where 


u* ~ K'x + K\z 


(11.2-17) 




K ( = —R-'G'P 


(11.2-18) 




K\ - -R-'G'P' lx . 


(11.2-19) 


The Riccati equation (11.2-14) becomes now 




« 

-p 


= PF f F’P - PGR-'G’P + Q 


(11.2-20) 


w 

~Pu 


= P 2i F + A’P 2l - P^GR-'G’P - C(H’H)-'H’Q 


(11.2-21) 


-P22 


= P 2i A + A’P 2l - P 2i GR-'G’P’ 21 






+ C{H’HY'H'QH{H’H)-'C 


(11.2-22) 



with the boundary conditions P{T) = 0, P 1X (T) = 0, and P 22 (T) = 0. The 
minimum index is 
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Fig. 11 .2-1 Regulator control of augmented system. 

V*(x(t 0 \ T) = x'(t Q )P(t 0 )x(to) + 2x'(t 0 )P' 2l (t Q )z(t Q ) 

+ z'(t 0 )P 22 (t 0 )z(t 0 )- (11.2-23) 

Figure 1.2-1 shows the augmented system (11.2-12) separated into its 
component systems (11.2-1) and (11.2-3) and controlled by use of linear 
state-variable feedback, as for a standard regulator. Figure 11.2-2 shows the 
same system redrawn as the solution to the servo problem. We observe that 
it has the form of a regulator designed by minimizing the index 

Hx(t 0 ), u(-), T) — I ( u’Ru + x'Qx) dt 

* to 

for the system (11.2-1) using regulator theory. In addition, there is an external 
input, which is the state z of the linear system (11.2-3) and (11.2-4). The 
feedback pan of the control is independent of A, C, and z(t g ). 




Fig. 11.2-2 Rearrangement of regulator system of Fig. 11.2-1. 




See. 11.2 



FINITE-TIME RESULTS 2S5 



The results to this point depend on Assumption 11.2-1, namely, that 
he s ate z is directly measurable. Certainly, if z is available, the servo problem 
is so ved. However, often in practice, only an incoming signal y is at hand 
or this case, a state estimator may be constructed with y as input and of 

“"r te 77 as r put ’ since ,he » air [a - c ) >p^ ri „ g p i„ o "is 

usfj the ~ ft, 77™ y , ° bS f vable ' The es,imator be constructed by 
at least if A ,„j ,! Chap ' ers 8 and 9, with f approaching z arbitrarily fast, 
and C are constant. The resulting system is shown in Fig. 1 1 ,2-3(a). 






Fig. 11 .2-3 Four forms for the optimal servo-system. 
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It is redrawn in Fig. 1 1.2-3(b) to illustrate that the state estimator and control 
law K\ can be combined to give a (dynamic) feedforward controller. 

Likewise, if the state x of the plant is not directly measurable, the 
memoryless linear state feedback may be replaced by a dynamic controller, 
which estimates the state x and then forms the appropriate linear transfor- 
mation of Ibis estimate, always assuming complete observability of H ]. 

Figure 11.2-3(c) shows this arrangement. Figure 11.2-3(d) shows one further 
possibility, where the estimate of x and z is carried out simultaneously in the 
one estimator; this arrangement may yield a reduction in the dimension of the 
linear system comprising the controller. We shall now summarize the optimal 
servo problem solution. 

Solution to the finite-time optimal servo problem. For the systems (11.2-1), 
(11.2-2), and (11.2-3), (11.2-4), and performance index (11.2-5), the 
optima] control w* is given by (1 1.2-17) through (11.2-22). The minimum 
index is given in (11.2-23). With Assumption 11.2-1 holding, the form 
of the optimal controller is indicated in Fig. 11.2-2(a). If only an esti- 
mate f of the state z is available, then an approximation to the optimal 
controller is as indicated in Fig. 11.2-3(b). The closer the estimate £ is 
to z, the better is the approximation. Further possibilities are shown in 
Figs. 11.2-3(c) and 11.2-3(d), where estimation of x is required. 

The tracking problem. It may be that the desired trajectory y(t) for all 
t in the range t 0 < t <Tis known a priori. If this is the case, the conduction 
of a state estimator to determine an estimate z of z is unnecessary. This repre- 
sents a considerable saving if y is, e.g., the output of a high-order, time- 
varying, linear system. We now define the optimal tracking problem and give 
its solution. 

Optimal tracking problem. Suppose we are given the n-dimensional linear 
system having state equations 

x = Fx + Gu x(t 0 ) given (1 1.2-1) 

y = H'x (11.2-2) 

where the m entries of y are linearly independent. Suppose we are also 
given an m vector y{t) for all t in the range t 0 < / < T for some times 
t Q and T with t 0 < T. The optimal tracking problem is to find the optimal 
control u* for the system (11.2-1), such that the output y tracks the signal 
y, minimizing the index 

V(x(t 0 ), u(-),T)= f {u'Ru 

J to 

+ x f [I - H(H 9 H)-'H']'Q l [I - H(H'H)- 1 H']x 
+ (y - y)'Q 1 (y - y)} dt (l 1.2-5) 




Sec. 11.2 



FINITE - TIME RESUL TS 257 



where Q x and Q 2 are nonnegative definite symmetric and R is positive 
definite symmetric. 

We first make the following temporary assumption. 

Temporary Assumption 1 1 .2-2. The vector y{t) for all t in the range t 0 < / < T 
is the output of a linear finite dimensional system 

z^Az (11.2-3) 

y = C'z (11.2-4) 

with the pair [A, C] not necessarily assumed to be completely observable. 

With this assumption holding, the optimal control u* is given using the 
optimal servo results (11.2-17) through (11.2-22) as 

u* = K'x + u ext (11.2-24) 

where 

u ex t = —R~ { G r b. (11.2-25) 

The matrix K is calculated as before from (11.2-18) and (11.2-20), and 
the vector b is the product ( P' 21 z ). Moreover, the minimum index V*, given 
from (11.2-23), may be written 

V*(x(t 0 ), T) - x'(t Q )P(t 0 )x(to) + 2x f (t 0 )b(t 0 ) + c(t 0 ) (1 1.2-26) 

where c replaces the term Z P 22 Z. 

The value of the matrices P 2l and P 22 and the vector z cannot be deter- 
mined independently unless the matrices A and C are known. However, the 
products b ~ P' 21 z and c = z'P 22 z can be determined directly from y(')> 
as follows. 

Differentiating the product ( P' 2l z ) and applying Eq. (1 1.2-21) for P f 2l 
and (11.2-3) for z, we get 

-P' 2l z-p' 2l z 

= FP' lt z + P' 2i Az - PGR~ X G'P\\Z - QH(H'H )~ 1 C'z 
- P'nAz 

= (F- GR-'G'P)'(P^z) - QH(H'H)- l y 

’With the boundary condition P 2l (T)z(T) = 0, following from P 2X {T) ~ 0. 
This means that with y(t) known for all t in the range t 0 < t < T, the term 
b can be calculated from the linear differential equation 

— £ ~ (F + GK')'b - Qx b(T) - 0 (11 .2-27) 



where, of course, 



x = H{H'H)' x y. 



( 11 . 2 - 8 ) 
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The optimal control law (11.2-24) and (11.2-25) is therefore realizable 
without recourse to using z, or an estimate z of z: Eq. (1 1.2-27) is solved, back- 
ward in time, to determine b(t ), which is then used in the optimal control law 
implementation. Matrices A and C do not play any role in determining m*. 

An equation for c(«) is determined by differentiating c ~ z f P 22 z as 
follows : 

^(z'F 22 z) = z'P 22 z + 2z'P 22 z 

= z'P 2l GR-'G'P' 2l z - z' C(H'H) ~ l H'Q H(H'H) ~ l C'z. 

Using the equations (11.2-4) and (11.2-8) and the identifications b = P' 2l z 
and c = z'P 22 z i we have that c(-) is the solution of the differential equation 

c = b'GR~ l G'b — xQx c(T) = 0. (11.2-28) 

We observe by using (11.2-27) and (11.2-28), that the matrices A and C do 
not play any role in determining V * from (11.2-26). 

Since the differential equations for b{ •) and <:(.•) can, in fact, be solved 
without Temporary Assumption 11.2-2, we indicate in outline a procedure 
for verifying that the control given by (11.2-24) and (11.2-25) is, in fact, 
optimal without Assumption 11.2-2. With u* defined by (11.2-24) and (11.2- 
25), but not assumed optimal, and with V*[x(t 0 ), T] defined as in (11.2-26) — 
again, of course, not assumed optimal — Problem 1 1.2-1 asks for the establish- 
ing of the following identity: 

V(x(t d ), w(*), T) = f (u — u*)'jR(w — w*) dt 

J to 

+ V*(x(t 0 \T). (11.2-29) 

Here, w(*) is an arbitrary control. Optimality of w* and V* is then imme- 
diate. The preceding results are now summarized. 

Solution to the finite-time tracking problem . For the system (11.2-1) 
and (1 1.2-2), and performance index (1 1.2-5), with the desired trajectory 
y(t) available for all t in the range t Q < t <T, the optimal control u* 
is given f rom 

u* = ~R l G'(Px + b) 

[see (11.2-24), (11.2-25), and (11.2-18)], where P ( •) is the solution of 
-P = PF + FP - PGR-'G'P + Q P(T) = 0 
and £(•) is the solution of 

-5 = (F - GR-'G'P)'b + QH(H’H)~ x y b(T ) = 0 
[see (11.2-27) and (11.2-8)]. The minimum index is 

T) = x’(t 0 )P(t 0 )x(t 0 ) + 2x’(t 0 )b(t 0 ) + c(t 0 ) (1 1 .2-26) 
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Fig. 11.2-4 Optimal tracking system. 



where c(? 0 ) is determined from 

t = b'GR-'G'b - y\H'H)~'H'QH(H'H)-'y c{T) = 0. 

The optima] controller is as indicated in Fig. 11.2-4. 

The model-following problem. The model-following problem we shall 
consider is one that is easily reduced to a servo problem. 

Optimal model-following problem. Find a control u* for the linear system 

x = Fx + Gu x(t 0 ) given (11.2-1) 

y = H'x (11.2-2) 

which minim zes the index 

V(x(t 0 ), u(- ), T) = f [u'Ru 

J to 

+ x'[I - H{H'H)- X H’]'Q X [I - H(H'H)- 'H’]x 

+ {y-y)'Qz{y-y)}dt. (l 1.2-5) 

In (11.2-5), Q x and Q 2 are nonnegative definite symmetric, R is positive 
definite symmetric, and y is the response of a linear system or model 

ii = A 1 z 1 + B x r z t (t Q ) given (11.2-30) 

y = C\z { (11.2-31) 

to command ir.puts r, which, in turn, belong to the class of zero input 
responses of the system 

z 2 = A 2 z 2 z 2 (t 0 ) given (1 1.2-32) 

r=C' 2 z 2 (11.2-33) 

as indicated in Fig. 11.2-5. 



The case when the command signal r is known a priori may be solved by 
direct application of the tracking problem results; it is left to the reader 
(see Problem 11.2-4). 

The two systems, (11.2-30) and (11.2-31), and (11.2-32) and (11.2-33), 
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command desired 

signal trajectory 




command 

generator 



model 



Fig. 1 1 .2-5 Desired trajectory for model-following problem. 



together fo;*m a linear system of the form of 

z = Az 



where z = \z\ 




z'J and the matrices A and C' are given from 




C - [C\ 0]. 



(11.2-3) 

(11.2-4) 

(11.2-34) 



For the case when z, and z 2 are available, the equations for the solution to the 
model-following problem are identical to those for the servo problem, with 
A and C givm by (11.2-34). In case of nonavailability of z 2 , state estimation 
is required, again in the same manner as in the servo problem. Figure 1 1.2-6 




Fig. 11.2-6 Model-following system. 



shows a moc el-following system with a feedforward controller consisting 
of a model to give z l9 and an estimator to give z 2 = z 2 . 

Problem 11.2-1. Show that the index (11.2-5), or (11.2-6), may be written 
in the form of (11.2-29) where u* is given from (11.2-24) and (11.2-25) and V* is 
given from (11.2-26). What happens if the term [x'(T) — jc'(r)]D[jt(r) — x(T)] is 
added to the performance index as a final-time cost term? Give the solution to the 
tracking problem for this case and verify your solution. 

Problem T 1 .2-2. Derive the tracking problem results directly, using the Hamil- 
ton-Jacobi theory of Chapter 2 (i.e., without using the regulator theory results). 
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Problem IT. 2-3. Extending the ideas of Chapter 10, give the form of the 
optimal servo and tracking systems if a term u'Zu is included in the integrand of the 
performance index (11.2-5). 

Problem 11 .2-4. Give two forms of solution to the model -foil owing problems 
as stated in the sijction, for the case when the command signal r is known a priori 
and construction of a state estimator for the command signal generator is out of the 
question. 



11-3 INFINITE-TIME RESULTS 

The finite-time results of the previous section yield servo systems, track- 
ing systems, and model-following systems, which consist of a standard opti- 
mal regulator with an appropriate external input. We expect that by extending 
these results to the infinite-time case, the various systems will consist of an 
infinite-time standard optimal regulator, again with an appropriate external 
input. We recall that the optimal regulator for the infinite-time case has some 
very desirable engineering properties, such as the ability to tolerate certain 
classes of nonlinearities in the input transducers. For the case when the plant 
is time invariant and the parameters of the performance index are time 
invariant, the optimal regulator has a controller that is also time invariant. 
This is, of course, another desirable property from the engineering point of 
view. Therefore, the infinite-time servo, tracker, or model follower can be 
expected to possess a number of desirable properties, such as those just 
mentioned. 

In this section, we restrict attention to plants and performance indices 
with time-invariant parameters, and we extend the finite-time results of the 
previous section to the infinite-time case. The aim is to achieve systems that 
track an input signal, and that also have the property that when the incoming 
signal is removed, the system has the same desirable properties as the standard 
optimal regulator. In particular, we require that the optimal system have time- 
invariant controllers. 

We shall first consider the case of the completely controllable plant 

x^Fx + Gu ( 11 . 3 - 1 ) 

y = H'x (11.3-2) 

and the performance index 

Wo), «(•)) = lim f {u’Ru + (y - y)'Q(y - y)} dt (11.3-3) 

T~*oe> J t 0 

where the matrices F. G, H , R , and Q are constant. The vector y is, as pre- 
viously, the desired output trajectory. For the sake of simplicity, we have 
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omitted the “smoothing” term in the index. We also assume that the pair 
[F, D] is completely observable for any D satisfying DD f = HQH'. We now 
discuss an important special case. 



Infinite-time cases completely reducible to regulator problems. Consider 
first the case when the desired output trajectory y is derived from an asympto- 
tically stable system, and is given by 



i = Az, z(t 0 ) given (11.3-4) 

y — Cz. (11.3-5) 

(We shall later remove the assumption of asymptotic stability.) In the 
previous section, we observed that the finite-time servo problem could be 
reformulated as a regulator problem for an augmented system 



1 




0 


• 

_ z _ 




1 

0 
1 




(11.3-6) 



Since [F, G] is completely controllable, and since z = Az is asymptotically 
stable, there is no trouble at all in extending the finite time results of the pre- 
vious section to the infinite-time case. The limiting solution of the relevant 
Riccati equation exists, the optimal controller is time invariant, and the 
resulting system consists of a standard time-invariant optimal regulator with 
an external input. The complete observability condition ensures that the sys- 
tem is asymptotically stable. For this special case, then, the objectives set 
out at the beginning of this section have been achieved. Problem 11.3-1 asks 
that the details for this case be developed. 

We consider now the case that arises when the desired trajectory y 
is not asymptotically stable. Without loss of generality, we assume that y 
is the outpui: of the completely observable system 




y = c\z t 



0‘ 



+ C 2 z 



2^2 



Oo) 

-^2(^0)- 



given 



(11.3-7) 

(11.3-8) 



where the real parts of all the eigenvalues of A t are nonnegative and the 
real parts of all the eigenvalues of A 2 are negative. In other words, none of the 

states z, are asymptotically stable and all the states z 2 are asymptotically 
stable. 

To consider this more general servo problem as a regulator problem, 
we introduce the following assumption. 



Assumption 11.3-1. The nonasymptotically stable contribution to y — viz., 
C\z u is a zero-input response of the plant (11.3-1), (11.3-2). (Afote; This implies 
that the eigenvalues of F include the eigenvalues of A x .) 
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Assumption 1 1 *3-1 implies that y can be regarded as the output of the 
following system : 

m onvi rwi ... 



Z 3 (‘o) 



given 



(11.3-9) 



y = H’z 3 + C' 2 z 2 , (11.3-10) 

where z 3 (t 0 ) is appropriately chosen. This, in turn, means that the servo 
problem is once again a regulator problem where the plant 



0 A 



is to be controlled to minimize the index 



(11.3-11) 



V(x(t 0 ), z 3 (t 0 ), z 2 (t 0 ),u(-)) 



lim f \u' 

T — «/ o I 



Ru + 



X ~ Z- 



-a 



X ~~ Z- 



Problem 11.3-2 asks that the details for this case be worked out. 

Once again, the aims as stated at the beginning of this section have 
been achieved; in essence, the servo problem has been reduced to the regu- 
lator problem. The difficulty is simply that the key assumption, 11.3-1, is 
very restrictive. However, one important example of a case in which it is 
satisfied occurs when y is a step function and the plant (11.3-1) and (11.3-2) 
is a single-input, single-output, “type 1” plant — i.e., the associated transfer 
function has a single pole at the origin (see Problem 11.3-4). But if in this 
example y were a ramp input instead of a step function, then Assumption 

11.3- 1 would not be satisfied unless the plant were first augmented with an 
integrator at its input. This suggests, for the case when Assumption 11.3-1 
is not satisfied by the original plant, that compensation {either series or feed- 
back) be included so that the resultant modified plant satisfies Assumption 

11.3- 1. 

As an example of the application of this principle, we shall now consider 
the case when a plant (11.3-1) and (11.3-2), with the property that Fhas no 
zero eigenvalues (i.e., jFis nonsingular), is to follow as closely as possible a 
step function input > . We first augment the plant with integrators at the input 
and then consider the control of the augmented system 



F G 



0 0 u 



(11.3-12) 



to minimize the index 



F(x(? 0 ),u(f 0 ),ti(.)) = lim \ T {u’Rii +(y-y)'Q(y-y)}dt. (1 1.3-13) 



Notice that this servo problem, for the case y = 0, reduces to the regulator 
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problem with derivative constraints given in the previous chapter. This servo 
problem can be solved by using the regulator theory of the last chapter; 
the details are requested in Problem 11.3-3. The optimal servo can be shown 
to consist of an optimal regulator having the form of those discussed in 
Chapter 10, with an external input signal. Once again, therefore, a method is 
given that achieves the objectives as stated in the introduction to this section. 

So far, we have discussed servo problems completely reducible to regu- 
lator problems. It is straightforward to apply these ideas to the corresponding 
model-following problems. For the tracking problem case, if the desired 
trajectory is the output of a finite dimensional linear system, then the preced- 
ing results can be interpreted to give a solution to the tracking problem. 
Problem 1 1.3-4 is concerned with such a situation. 

We now move on to consider what happens if we extend the finite-time 
result of the previous section to the infinite-time case without requiring that 
the resulting performance index be finite. Oddly enough, it turns out that 
results of engineering utility can still be obtained. 

Direct extensions of finite-time results to the infinite-time case. For the 

system (11.3-1) and (11.3-2), performance index (11.3-3), and desired tra- 
jectory y given by (11.3-4) and (11.3-5) (the servo problem), a control u 
may be found by taking the limits of the finite time results for the optimal 
control, provided, of course, that these limits exist. (Note that u is not neces- 
sarily optimal, but is still perhaps useful.) Assuming for the moment that 
these limits exist, we have that 

u = K'x -j~ K\z (11.3-14) 

where 

K' = ~R~ ] G'P K\ = -Rr x G'P lx (11.3-15) 

P = lim P(t, T) P 2l =lim P Zi (t,T). (11.3-16) 

7'-»oo y — .oo 

The equations for P(*, •) and P 21 (*, •) are > as * n the previous section, given 
from 

~P - PF + F'P - PGR^G’P + HQH' P(T , T) = 0 (11.3-17) 

= P 2l F+A'P 21 - P 2l GR~'G'P-CQH ' P 2l (Z T) = 0. 

(11.3-18) 

We have, from the infinite-time regulator theory of Chapter 3, Sec. 3.3, 
that since the pair [F, G] is completely controllable, the limiting solution P 
of the Riccati equation (11.3-17) exists. Theory concerning the existence of 
F 21 is not so straightforward. 

To find conditions that ensure the existence of F 21 , we rearrange the 
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linear differefttUl equation (11.3-18) using the relation (11.3-15) as follows: 

~Pu = ^'21 (P + GK’) + A'P 2i — P 2I GR~‘G'(P - P) + CQH ' 

P U {T, T) = 0. (11.3-19) 

An expression for (P - P) is given later in the book in Chapter 15, Sec. 
15.2. This expression indicates that for any fixed t, (P(t, T) — P) decays expo- 
nentially toward zero as T approaches infinity. Kreindler [1] has shown, using 
a result of [4], that if and only if P 2i exists, the limit 

n 21 = lim IT 21 (r, T) (11.3-20) 

7* — ► CO 

also exists where II 2 ,(- > T) satisfies 

-Il 21 = II 21 (F + GK') + A' n 2I - CQH' n 21 (T, T) = 0. 

(11.3-21) 

Moreover, P 2l =l} 2i . In contrast to (11.3-19), the existence of limiting 
solutions of (1 1.3-21) is easy to check. The solution of (1 1.3-21) may be verified 
to be 



n 21 (t, T) = £ exp [A'{t - x))CQH' exp [(F + GK’)(t - r)] dx. 

(11.3-22) 

By inspections, we see, then, that fl 21 , and thus P 21 , will exist if and only if 
the following assumption holds . 

Assumption 11.3-2. The sum of the real parts of any eigenvalue of A and any 

eigenvalue of (F f GK') is negative. 

With Assumption 11.3-2 holding, a method has been indirectly outlined 
for achieving a servo system with the property that it consists of an optimal, 
time-invariant regulator with an external input. Since the controllers for the 
system are time invariant, the objectives as stated in the introduction to this 
section are achievec. Observe, however, that no indication has been given 
concerning any performance index being minimized. Unless Assumption 
1 1.3-1 holds, the infinite-time performance index is infinite. The only general 
optimal control interpretation of these results is that they are the limiting 
results of the finite-time optimal servo problem case. That is, they have 
properties very close to the optimal systems designed for a large terminal 
time T. 

One important observation is that the results of Chapter 4 may be 
used to select a value of Q in the performance index to ensure that Assump- 
tion 11.3-2 is satisfied. By choosing a sufficiently large positive definite value 
of Q , the real parts of the eigenvalues of (F — GK') may be guaranteed to be 
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less than any specified negative value. Clearly, the location of the eigenvalues 
of A will determine what this specified value is chosen to be. Problem 1 1.3-5 



asks that the details for such a design be developed. 




For the tracking problem, the control u is given from 






u = —R~'G'(Px + 1) 


(11.3-23) 


where 


b= lim b(t, T) 

f — >oo 


(11.3-24) 


with b( •, 


•) the solution of the linear differential equation 






-6 = (F + GK')'b - HQy b(T, T) = 0. 


(11.3-25) 



For the particular case when y is the output of a linear system (11.3-3) 
and (11.3-4) with Assumption 11.3-2 holding, then b = F' 21 z, and b is 
clearly well defined. For other situations, to determine if b is well defined 
would require a separate investigation for each y, unless y is asymptotically 
stable or is within a square integrable function of an output of a linear 
system. 

An interesting application for the preceding theory arises when the system 
is single input, single output, and when y is a step function. For this case, 
we have x = Fx + gu , y = h'x, Q = 1, R = p, k! = ~-p~ l g'P, b = [(F + 
gk')']~ l hy , and u — — — p' l h'(F + gk’)~ l g. The following results may 
now be verified. The steady-state output is y = [A'(f + gfc) *g] 2 ; the steady- 
state control signal is u = [1 + &'(F + g/0 _1 g][— + g/C)' ! g]; and the 
steady-state error is e — [/f(F + g^')' 1 ^] 2 — 1 = —[1 ~ A:'(F+ gk’Y'g ] 2 . 
For the case when F is singular (i.e., has a zero eigenvalue), then det 
[/— (F + gk’Y'gk'] = det[l — fc'(F + g/c') _1 g] = 0, and thus ti(F + g/c') _I g 
= zty, and y = j and u — 0. An interesting property of this tracking 
system is that as p approaches zero, the poles of the closed-loop transfer 
function approach a Butterworth configuration, except for some poles equal 
in number to the number of zeros of the transfer function. These poles 
approach the zeros of the transfer function. Also, as p approaches zero, the 
bandwidth of the system and the control energy required increase. A discus- 
sion of this phenomenon is given in [5] and in Chapter 5, Sec. 5.4. 

For the model-following problem on an infinite time interval, the servo 
results can be applied directly. 

We conclude this section with a fairly lengthy example, drawn from 
[6] and [7]. The example illustrates the design of a controller for the lateral 
motion of a B-26 aircraft. The main idea is to provide control to cause the 
actual aircraft to perform similarly to a model; the way this qualitative 
notion is translated into quantitative terms will soon become clear. 

The general equations governing the aircraft motion are the linear set 
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+ 



'0 

Ldr 

Ysr 

N 6r 



+ N' p Y Sr 



0 

U 

o 

N 





In these equations, <f> denotes the bank angle, ]3 the sideslip angle, r the yaw 
rate, d r the rudder deflection, and 5 a the aileron deflection. Of course, we 
identify x with [$ 0 /? r]\ etc. The quantities L p , etc.-, are fixed parameters 
associated with the aircraft. For the B-26, numerical values for the Fand G 
matrices become 





~0 1 0 

0 -2.93 -4.75 

0.086 0 -0.11 

0 -0.042 2.59 

~ 0 1 “ 

0 - 3.91 

0.035 0 

_-2.53 31 



0 “ 
-0.78 
- 1.0 
-0.39_ 



However, the dynamics represented by this arrangement are unsatisfactory. 
In particular, the zero-input responses are preferred to be like those of the 

model 



i = Az 




'0 

0 

0.086 

0.0086 



1 

-1 

0 

0.086 



0 0 

-73.14 3.18 

- 0.11 -1 
8.95 -0.49 



This model is derived by varying those parameters in the F matrix correspond- 
ing to aircraft parameters which could be physically varied. For this reason, 
the first and third rows of A are the same as the first and third row of F. 
The eigenvalues of A are 

- 1 .065, +0.00275, -0.288 ± ;2.94. 
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Although one is unstable, the associated time constant is so large that the 
effect of this unstable mode can be cancelled by appropriate use of an external 
nonzero input. 

To achieve actual performance resembling that of the model, we pose the 
quantitative problem of minimizing 

poo 

[u'u + (x — z)'Q(x — z)] dt. 

J 0 

Thus, we are considering a servo problem that differs from the standard one 
only by the specializations H = C = I. 

As we know, the optimal control becomes of the form 

u — K'x + K\z 

where K and K x are determined by the methods given previously. In practice, 
the control 

u = K'x + K\z + w ext 

would be used, where u ext denotes an externally imposed control. Likewise, 
in practice, the model equation i = Az would be replaced by 

i — Az + Gu ext . 

At this stage, we are faced with the problem of selecting the matrix 
Q in the performance index. From inspection of the performance index, 
it is immediately clear that the larger Q is, the better will be the following of 
the model by the plant. This is also suggested by the fact that large Q leads to 
some poles of the closed-loop system 

x = (F + GK')x 

being well in the left half-plane — i.e., leads to the plant with feedback around 
it tending to respond fast to that component of the input, K\z , which arises 
from the model. 

On the other hand, we recall that the larger Q is taken, the larger will 
be the entries of AT, and, for the aircraft considered, it is necessary to restrict 
the magnitude of the entries of K to be less than those of 




To begin with, a Q matrix of the form pi can be tried. Either trial and 
error, or an approximate expression for the characteristic polynomial of 
F + GK' obtained in [7], suggests that Q = 5/ is appropriate. This leads to 
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eigenvalues of F + GK' which have the values 

-0.99, -1.3, -5.13, -9.14. 

Larger Q leads to more negative values for the last two eigenvalues, whereas 
the first two do not vary a great deal as Q is increased. Comparison with the 
model eigenvalues suggests the possibility of further improvement, in view 
of the fact that the ratio of the nondominant eigenvalue of F + GK ' nearest 
to the origin to the eigenvalue of the model most remote from the origin is 
about 5. On the other hand, the gain matrix K associated with Q = 51 is 



-2.53 


-2.21“ 


-0.185 


-1.83 


1.58 


0.7 


-2.34 


-0.01 



and at least one of the entries (the 2-2 one) is near its maximum permissible 
value in magnitude. 

This suggests that some of the diagonal entries of Q should be varied. 
To discover which, one can use two techniques. 

1 . One can plot root loci for the roots of F + GK\ obtained by varying 
one q ir Variations causing most movement of the roots leftward 
and simultaneously retaining the constraints on K can be determined. 

2. One can examine the error between the model state z(t) and the 

plant state x(/) obtained for several initial conditions, z(0) = x(0) 
-[10 0 0]', [0 1 0 0]', [0 0 1 0]' and [0 0 0 1]', 

for example, using the design resulting from Q — 51. One can then 
adjust those diagonal entries of Q which weight those components 
of (z — x) most in error. 

Case 2 leads to the greatest errors being observed in (z 2 — x 2 ) and (z 4 
— x 4 ). This suggests adjustment ofq 22 and/or q 44 , and this is confirmed from 
(1). However, adjustment of q 21 causes the 2-2 entry of K to exceed its maxi- 
mum value. On the other hand, adjustment of # 44 , from 5 to 20, proves satis- 
factory. The new eigenvalues of F + GK’ become 

-0.908, -0.66, -9.09, -11.2, 

and the gain matrix K is 

~ — 0.201 -2.23 “ 

-0.185 -1.83 
1.42 0.164 

-4.42 — 0.264_ 

For completeness, we state the feedforward gain matrix K x associated 
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with the model states. For Q = diag [5, 5, 5, 20], this is 



K 



i 



0.101 

0.344 

-2.153 



2.045 

2.172 

-1.54 



5.61 2.42 



Although the model is unstable, the sum of any eigenvalue of the model 
matrix A and the matrix F 4- GK' is negative, which as we know guarantees 
the existence of K v . 



Problem 1 1 .3-1 . Suppose we are given the time-invariant completely control- 
lable plants (11.3-1) and (11.3-2), and the performance index (11.3-3). We are also 
given that the desired trajectory y is the output of a linear, time-invariant, asymp- 
totically stable system 

2 — Az 
y = C'z. 

Apply regulator theory to determine a time-invariant optimal controller for this 
system. 



Problem 1 1 .3-2. Suppose we are given the time-invariant, completely control- 
lable plant (11.3-1) and (11.3-2), and the performance index (11.3-3). We are also 
given that Assumption 11.3-1 is satisfied. Apply regulator theory to determine a 
time-invariant optimal controller for this system. How are the results modified for 
the case when the index is given by 

V(x(t 0 \ «(•)) = lim T {u'Ru + x[l - Q { [I - H(H'H)~ l H / ]x 

T~*°° J to 

+ (y ~ y)'Qi(.y — y)dtl 

Problem 11.3-3. Suppose you are given the plant (11.3-1) and (11.3-2), where 
F has no zero eigenvalues, and also the performance index (11.3-13). Develop an 
optimal servo system for the case when the incoming signal y is a step function. 
Where in the system can large sector nonlinearities be included without affecting 
the stability properties of the system? 

Problem 1 1 .3-4. Suppose you are given the system (11.3-1) and (1 1 .3-2) where 
the output is to track y = constant, minimizing the index (11.3-3). 

1. Develop a tracking system for the case when F is singular (i.e., F has a zero 
eigenvalue) by reducing the problem to a regulator problem. 

2. Develop a tracking system using tracking theory and discuss what happens 
when F is singular. Also, set up the tracking scheme so that the input is y 
and there is no feedforward controller. 

Problem 11.3-5. Suppose you are given a plant (11.3-1) and (11.3-2) and an 
incoming signal y given from (1 1 .3-4) and (1 1 .3-5), where the real parts of the eigen- 
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values of A are all less than a positive constant a. Show how a finite-time perfor- 
mance index may be set up so that when minimized, the limiting optimal control, 
as the time interval of interest approaches infinity, will be finite. (Hint: Use the ideas 
of Chapter 4 to ensure that Assumption 11.3-2 is satisfied.) 

Problem 11 .3-6. For the infinite-time servo problem, where the performance 
index is always infinite for all controls, is it possible to set up the problem using 
an index 

V(x(t Q \ «(•)) = lim ~ f [u'Rii + (y — y)'Q(y — y)} dt 

1 J to 

so that the difficulties associated with an infinite index do not occur? Discuss this 
case. 
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CHAPTER 



OPTIMAL REGULATORS WITH 

DUAL-MODE CONTROL 



12.1 THE BANG-BANG MODE AND LINEAR MODE 



In this section, a qualitative description is given of some optimal and 
suboptimal regulators with dual-mode control. The section serves as an 
introduction to the next section, which applies standard regulator theory to 
give useful results that may be used in the design of both optimal and sub- 
optimal dual-mode regulators. The emphasis in the chapter is on the applica- 
tion of the results of earlier chapters to the problem of designing dual-mode 
regulators rather than on giving an exhaustive treatment of this subject for 
its own sake. Since an understanding of relay systems is essential to the 
development of the topic, a review of some of the relevant results of Chapter 
6, Sec. 6.2, is now given. 

A relay system having the form shown in Fig. 12.1-1 and having the 
closed-loop state equations 

* = Fx + g sgn (k'x) (12.1-1) 

has two modes of operation. In one mode, the bang-bang mode , the input to 
the relay is either a fixed positive or fixed negative value, and any sign 
changes occur instantaneously. In the other mode, earlier termed the chatter- 
ing mode but also known as the sliding mode , singular mode , or linear mode , 
the relay input is zero and the output chatters between its maximum and 
minimum output value with an average sliding value that could alternatively 
be realized by a linear state-feedback control law. This latter point is of 
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(c) 

Fig. 12.1-1 Relay dual-mode control system. 



particular interest to us in this chapter inasmuch as we are fundamentally 
concerned with linear control theory. Typically, for a large initial state, the 
regulator will operate first in its bang-bang mode and then, when the states 
are sufficiently small, it will enter the chattering mode and remain in this 
mode even on reaching the zero state. Figure 12.1-1 shows the form of such 
a relay system, a typical state trajectory and the corresponding control input 
Note ,h.t ,he control indicated in Fig. 12,I-l(c) for the chattering mode il' 
the average control rather than the true control. 

For the chattering mode, the states obey the equations 



k'Fx | < k'g 



( 12 . 1 - 2 ) 
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and, if k'g is nonzero. 




The bounded hyperplane defined by (12.1-2) will be termed the singular 
bounded hyperplane throughout the chapter. The singular strip is that portion 
of the singular bounded hyperplane with the property that solutions of (1 2. 1 -2) 
starting in the singular strip remain in the singular bounded hyperplane. 
For the case when x is a two-dimensional vector, the singular strip is the 
entire singular bounded hyperplane. For the case when x is a vector of size 
greater than 2, the relay system (12.1-1) may operate in the bang-bang mode 
and the chattering mode alternately until the state trajectory enters the sin- 
gular strip. Once in the singular strip, of course, the trajectory will remain in 
the singular strip. 

One quite significant property of the relay systems just described is that 
chattering mode performance is not at all affected by the introduction of a 
wide class of nonlinearities into the input transducers of the plant. That is, 
for the system 

x = Fx + gfi sgn ( k'x ), (12.1-4) 

where ft is any positive time varying gain, the chattering mode trajectories 
are still given by (12.1-3), as in the case of system (12.1-1). Of course, the 
bang-bang mode trajectories will depend on /?, but, as discussed in Chapter 
6, the (nonglobal) stability properties of relay systems are usually dependent 
only on the stability properties of the chattering mode. 

A rather obvious but significant property of the relay system is that the 
plant input is prevented from exceeding a specified upper limit. Of course, 
a saturation device rather than a relay at the plant input may be quite 
adequate in achieving the same purpose. Note that here, again, such a system 
operates dual mode. There is a linear mode and a saturation mode , as indi- 
cated in Fig. 12.1-2. This form of dual mode control leads on to a more gen- 
eral form, which we shall now discuss. 

A dual-mode control system may be constructed with a switch that 
connects either of two controllers into the feedback loop at any one time. 
One controller implements a linear feedback law, and the other operates in 
a bang-bang mode, with the times of switching dependent on the location of 
the system trajectory in the state space. Figure 12.1-3 illustrates this more 
general form of control. 

If an on-line computer is available, quite sophisticated forms of dual- 
mode control may be realized, and it may even be that in the case when the 
plant is highly nonlinear or the application is quite critical, only a sophisti- 
cated approach can achieve the desired results. As an example (later to be 
studed in some detail), we pose the problem of finding to a high order of 
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(0 



Fig. 12.1 -2 Dual-mode regulator with saturation, 
precision a control u for the plant 

x = Fx+gu (12.1-5) 

which will minimize the index 

/* OQ 

v ix(t 0 ), «(•), to) = (x'Qx) dt (12.1-6) 

with Q nonnegative definite, subject to the inequality constraint 

(12.1-7) 

The application of optimal control theory to this singular optimal control 
problem yields the result that the optimal control is a bang-bang law over 
most of the state space with the switching surface being a hypersurface, which 
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(a) 





Fig. 12.1-3 Optimal dual-mode control. 

is, in general, not linear. However, there is a certain portion of the hypersur- 
face in the vicinity of the state space origin that is a bounded hyperplane 
(i.e., a linear hypersurface) known as the singular strip , for which the optimal 

control u* satisfies \u*\ < 1. Once the state trajectory enters the singular 
strip, it remains in the singular strip. 

To implement the optimal control law for the preceding problem 
requires a computer to store the characteristics of the hypersurface to decide 
when to switch from w* = +1 to w* = — 1, and vice versa. But for the 
mode of operation for which |w*| < 1, a computer is not needed. In fact, 
the control law is linear and is therefore readily realized as for the standard 
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regulator problem. It may also be realized by a relay in its chattering mode. 

There is clearly a disadvantage in using the described form of dual- 
mode control — viz., an on-line computer is required. Unless a computer is 
available for other purposes as well, or unless it is vitally important to have 
precise optimal performance, this approach is probably not warranted, espe- 
cially since by using other, more straightforward means nearly optimal per- 
formance may be achieved. 

In fact, in the example, by using a simple relay controller, it is possible 
to achieve optimal performance for a region in the vicinity of the singular 
strip and nearly optimal performance outside this region. (The control law 
associated with the relay operating in its chattering mode is the same as the 
optimal, singular-strip control law.) When the relay controller is used, the 
optimal switching hypersurface is approximated by a hyperplane, which is 
an extension of the singular strip as indicated in Fig. 12.1-4. The singular 



Xa 




strip for the relay system (defined, we remember, as the set of points on the 
switching surface such that if chattering occurs at any such point, it occurs 
thereafter), will, of course, coincide with the singular strip for the optimal 
dual-mode system (i.e., the set of points such that the optimal control law 
takes on the linear form). Figure 12.1-5 indicates a region of the phase plane, 
which is the set of all initial conditions for the relay system that result in 
optimal state trajectories. For initial conditions outside this region, the state 
trajectories are suboptimal. 

There are two important differences between the optimal system and the 
relay system that reflect in the state trajectories. The first difference is that the 
optimal switching surface is nonlinear, whereas the relay system switching 
surface is linear. Figure 12.1-6 illustrates the differences between the optimal 
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Fig. 1 2.1 -5 State space region for which relay system is optimal. 




Fig. 12.1-6 Optimal and suboptimal trajectories. 

S Ct “ ^ Suboptimal ones of the relay system due to this fact. 

rtt, °H-fr the C0 . ntr ° S for the two cases are different, there is, in fact, very 
little difference between the performance indices V= j °°{x'Qx)dt for the 

two cases. The second difference is that chattering with the relay will occur 

in a singular bounded hyperplane, which is generally more extensive than the 

singular strip. The relay may switch from its chattering mode to the bang- 




Sec. 12.1 



THE BANG-BANG MODE AND LINEAR MODE 279 



bang mode. This never happens for the optimal system unless there is an 
external disturbance. This difference is difficult to illustrate because it does not 
occur for a second-order system. But once again, the control may be signif- 
icantly different without the performance indices differing very much. 

As mentioned in Chapter 6, it might be that the engineer requires a con- 
trol system with the same properties as the preceding suboptimal relay 
system, but for reliability or some similar consideration cannot use a relay 
in its chattering mode. For this case, a relay can be used until the com- 
mencement of chattering; then, a linear control law (say, k\x) can be switched 
in so as to achieve the same control law as the relay in its chattering mode. 
We observe that if the control law k\x is changed to (k\ + yk')x for some y, 
then control is independent of y as long as the trajectories lie within the singu- 
lar bounded hyperplane, where the condition k'x = 0 is satisfied. Clearly, if 
there is some disturbance of the trajectories from the singular bounded hyper- 
plane so that k’x is no longer zero, then the effect of the control law u — 
(fdx + yk f )x will depend on the value of y. It is therefore important to choose 
an appropriate value of y to be used for any particular application, since, in 
practice, this law will be used in a region of the state space on either side of 
the singular bounded hyperplane. 

Whatever the linear law used, it is necessary to consider how reversion 
to the bang-bang mode can be achieved. It might be that once the linear law 
has been switched in, it would be used even in the presence of a disturbance 
as Jong as the control satisfied \u \ < 1. When \u \ = 1, the relay would be 
switched in to bring the system back to the singular bounded hyperplane. 
On the other hand, a controller might monitor the magnitude of k'x, and 
when this exceeded a certain value, the relay with input k'x would be used as 
the control law to return the state trajectory to the singular bounded hyper- 
plane. Figure 12.1-7(a) and 12.1 -7(b) indicates the alternative linear mode 
control arrangements. Passage from the linear mode to the bang-bang mode 
occurs at the boundaries of the shaded regions of the figures, and passage 
from the bang-bang mode to the linear mode occurs only on the singular 
bounded hyperplane. 

The preceding remarks emphasize the scope for ingenuity in arriving 
at the best form of dual-mode control for any application. 

A particularly attractive form of control law for the linear mode of 
operation of a dual-mode control scheme would be a law that minimized a 
quadratic index (uu + x’Q x x) dt , such as arises in the standard regulator 
problem. This would mean that in the linear mode of operation, the system 
would have all the desirable engineering properties associated with the 
standard regulator. We show in a later section that such a form of control 
law can be found by choosing the value of y in the control law u = ( k\ + yk')x 
to be sufficiently large. 

It might be argued that if the minimization problem posed in the first 
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Fig. 1 2.1 -7 Possible regions for linear-mode control after singular 
strip has been reached. 



place is the minimization of an index 

f oo 

V = j (u u 4 ~ -X Q j x) dt 
J t 0 

for some nonnegative definite Q { , subject to the basic system equation (12.1-5) 
and control magnitude constraint (12.1-7), then a system with the desirable 
properties discussed would be realized immediately. However, for this mini- 
mization problem, the optimal linear law is applicable in a region of the 
state space that is not confined to a hyperplane, and there is a region of the 
state space for which the optimal control law is nonlinear but not bang 
bang, as illustrated in Figure 12.1-8. For the case when Q { is chosen as pQ 
for a constant /?, and then p approaches infinity, the linear control law region 
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Fig. 12.1-8 Control regions resulting 
from minimizing a quadratic index with 
an upper bound on the control magni- 
tude. 




reduces to the singular strip of the original singular minimization problem, and 
the nonlinear region evanesces. But then the theory of the original minimiza- 
tion problem [Eqs. (12.1-5), (12.1-6), and (12.1-7)] is needed to study this case. 

In the next section, we show that standard regulator theory can be applied 
to determine at least a part of the optimal solution for the problem of mini- 
mizing the index (12.1-2), subject to the constraints (12.1-5) and (12.1-7). 
In particular, the singular strip is described and the optimal control is found 
when the trajectories lie within this surface. That is, we solve for the singular 
solutions or linear mode solutions of the singular optimal control problem. 
Since the theory developing the optimal control law for the bang-bang mode 
cannot be readily applied without an on-line computer, it will not be discussed 
in this chapter. (For further details, consult reference [1].) As part justifica- 
tion, we note again the two points discussed earlier: Implementation of a 
suboptimal law using a relay or dual-mode control agreeing with the singular 
optimal law in the linear regime, but not the bang-bang optimal law in the 
bang-bang regime, leads to a value of the performance index not far different 
from the optimal value, and leads also to systems with pleasing engineering 
properties. 

In the third section of the chapter, we consider the properties of dual- 
mode systems constructed by using the results of Sec. 12.2. Of particular 
interest is the ability to accommodate nonlinearities without introducing 
instability. A fourth section is included, which discusses the optimal control 
of a system subject to the constraint | u | < 1 . This is an application of the 
results of the previous sections together with the results of Chapter 10. 
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Much of the material of this chapter is taken from references [2] through 
[5]. For further background reading, references [6] through [10] are suggested. 



12.2 LINEAR MODE SOLUTIONS TO THE SINGULAR 
OPTIMAL CONTROL PROBLEM 



[n this section, the regulator theory of the preceding chapters is applied 
to obtain the linear mode control laws or, equivalently, the singular solutions 
of what is known as the singular optimal regulator problem. A transforma- 
tion is used to convert the problem of interest into a form of the standard 
regulator problem, and the standard regulator results are then interpreted 
so as to give the solution to the original problem. The results will be applied 
in subsequent sections to the design of dual-mode control systems consisting 
of a bang-bang mode and a linear mode. As a first step, we express the prob- 
lem in mathematical terms. 

The singular optimal regulator problem . We consider, as before, systems 
having state equation 

x = Fx + Gu (12.2-1) 

where F and <7, if time varying, have continuous entries. As before, jc 
is an n vector, the state, and u is an m vector, the input. Constraints are 
imposed on the system inputs, which may be expressed without loss of 
generality as simply 

|n,| < 1 / = 1, 2, . . . , w. (12.2-2) 

The performance index of interest is 

F(x(/ 0 ), i/(«), t 0 ) = f (xQx) dt (12.2-3) 

J to 

for some finite T>/ 0 , where Q is a nonnegative definite symmetric 
matrix. The problem is to find the optimal control law */*(•), which 
applied to the system (12.2-1) minimizes the index (12.2-3) subject to the 
inequality constraint (12.2-2). The infinite-time problem is defined by the 
limiting case T — > oo. 

For the preceding regulator problem, the optimal controller provides 
dual-mode control at each input; one mode, the bang-bang mode, operates 
when an input u,(0 takes on the values +1 or — 1, whereas the other mode, 
the singular mode, occurs when | u { \ < 1. The latter is also referred to as a 
linear mode , because it may be realized by a linear control law, or as a chat- 
tering mode , because it may be realized by a relay with its output chattering 
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between +1 and —1. For this condition, the relay input is zero, and, as 
mentioned in the previous section, the terminology singular mode is used. 

The preceding problem is an interesting mathematical one in its own 
right, but more important for us here is its engineering significance. We recall 
that for a standard regulator, the integrand of the performance index is 
(x'Qx + uRu) with R positive definite. The term uRu is included so that 
the optimal control «* is bounded within reasonable limits. Here, the limits 
are set directly via the inequality constraint (12.2-2), and thus there is no 
need to include a uRu term in the index for input limitation purposes. The 
fact that limits are set directly means that from an engineering point of view 
the singular optimal regulator problem is better posed than that for the 
standard regulator, provided, of course, that there is no restriction requiring 
the controller to be linear. 

As a first step in solving the optimization problem, we consider the 
minimization of the index (12.2-3), subject to the system equation (12.2-1) but 
not subject to the inequality constraints (12.2-2). Once this problem has been 
solved, we shall discard those solutions that violate the constraints on u. 

To solve for the singular solutions of this problem, we define new 
variables u x and x { through 

= u (12.2-4) 

x x = x - Gu v (12.2-5) 

and new parameter matrices 

H^QG R = G'QG G X =FG-G. (12.2-6) 

The variables u and x in (12.2-1) may be replaced by u x and x x + Gw,, respec- 
tively, which leads to 



= Fx x +G x u x . (12.2-7) 

Notice that (12.2-4) does not specify w, uniquely, but only to within a con- 
stant. Nevertheless, whatever w, is derived, Eq. (12.2-7) holds. Notice also 
that Eq. (12.2-1) follows from (12.2-7) and the definitions (12.2-4) and (12.2-5). 

The performance index (12.2-3) may also be expressed in terms of x x 
and w, as follows: 

V(x(t Q \ w(-), 1 0 ) - V(x x (t 0 ) 9 w,(* ), t 0 ) under (12.2-4) and (12.2-5) 

= \ (AQx i +2x r i Hu i +u\Ru x )dt 

= f [A{Q - HR^Hjx, + (w, + R-'H'xJR 

J to 

X (w, + R-'H'x x )]dt. (12.2-8) 

We now see that the transformations (12.2-4) and (12.2-5) have converted the 
singular minimization problem into the quadratic regulator problem of Chap- 
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ter 3, Sec, 3,4, at least for the case when R is positive definite. For the case 
when R = G'QG is not positive definite, the index will obviously not have the 
standard form, and further transformations are necessary. These cases are 
considered in Problem 12.2-1. 

From the way the two minimization problems are related, we see 
immediately that under the converting relations (12.2-4) and (12.2-5), the 
minimization of (12.2-8), subject to the constraint (12.2-7), is precisely the 
same as the minimization of (12.2-3), subject to (12.2-1). As already noted, 
the minimization of (12.2-8), subject to the constraint (12.2-7), is essentially 
nothing other than a standard regulator problem. 

Without further comment, we shall restrict attention to the case when 
R = G'QG is positive definite. (Note that when Q is positive definite, this 
condition is automatically satisfied if the inputs are independent or if there is 
one input. Even in the case when Q is singular, there are many possible G 
matrices such that R > 0.) 

We first claim that ( Q — HR~ X H') is nonnegative definite as a conse- 
quence of the nonnegativity of Q and the particular way in which H is defined. 
To see this, observe that x'Qx is nonnegative for all x. Thus, using the trans- 
formations (12.2-4) and (12.2-5) as before, the term 

[x\(Q- HR-'H')x 1 +{u x + R-'H'x { yR(u x + R~ l H , x l )\ 

is nonnegative for all x { and u x . For arbitrary x x , setting u v = — R~ l H'x x 
leads to the conclusion that x\(Q — HR~ i H')x l is nonnegative and then that 
( Q — HR { H') is nonnegative definite. 

Since R is positive definite and ( Q — HR~ l H ') is nonnegative definite, 
the standard regulator theory may be applied directly, as in Sec. 3.4, Chapter 
3. For the finite-time case, the optimal control uf is given by 

uf = K'x, (12.2-9) 

and the minimum index V* is given as 

v*(x,{t o), / 0 ) = A(t 0 )P(t 0 , T)x t (t 0 ). (12.2-10) 

Here, K is given from 

K'= -R~ l (G\P + //')> (12.2-11) 

and P(-,T) (where existence is guaranteed) is the solution of the Riccati 
differential equation 

-P ~ P(F — G.R-'H') +(F- G^R-'H'yP — PG X R~ 1 G\P 

+ (Q~ HR-'H ) P(T ; T) - 0. (12.2-12) 

Before interpreting these results in terms of the desired solution to the 
finite-time singular regulator problem, we shall perform some manipulations 
using the various equations and definitions to yield some simple relationships 
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between the various quantities hitherto defined. In particular, we shall estab- 
lish in the order listed the relationships (valid for all r): 

PG — 0 K'G = -/ V* — x'(f 0 )P(f 0 . 7>(f 0 ) * = 0. 

Postmultiplying both sides of the Riccati equation (12.2-12) by the matrix 
G gives 

~PG = P(F-~ G x R~ x H’)G + (F~ G { R-'H')'PG - PG X R X G\PG 
+ (0 - HR~ l H')G. 

Applying the definitions (12.2-6) (viz., R = H'G , H = 0G, and Gj = FG 
— G), gives immediately that 

~^r(PG) = (F' - //K-'G', - PGiR-'G'^PG. 

Now, since F(F, F)G ~ 0, the preceding differential equation in ( PG ) has the 
solution 

PG — 0 for all t. (12.2-13) 

This is the first of the simple relationships. 

Postmultiplying both sides of Eq. (12.2-11) by the matrix G gives that 

K'G = - R'{G\PG + H'G). 

Applying the result PG = 0 and the definitions (12.2-6) — in particular, 
R ~ H'G — yields the second simple result 

K'G = -/, together with K'G + K'G - 0. (12.2-14) 

For the case when the control u x is, in fact, the optimal control wf, then 
Eq. (12.2-5) becomes 

x, - x - Guf (12.2-15) 

and the minimum index K* given from (12.2-10) may be written as 
v *(x { (t 0 ), t 0 ) = M>o) - OM'o) - G(r 0 )uf(? 0 )]. 

Since PG = 0 from (12.3-13), this becomes a function of only jc(/ 0 ) and t Q 
as follows; 

V*(x(to\ U) - xy 0 )P(t 0 , T)x(t 0 ). (12.2-16) 

Th is is our third simple result. Finally, for the case when u x - uf, we have 
that 

K'x = K'x { + K'Guf - K'x x + K'G K'x x . 

Using the result that K'G — —/[see (12.2-14)], we reduce this equation to our 
fourth simple result: 

K'x = 0. (12.2-17) 

This result may seem somewhat paradoxical. A first glance at the earlier 
remarks would seem to indicate that, starting with an arbitrary x(t Q ), not 
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necessarily satisfying (12.2-17), one could set up an associated x { (t Q ), solve 
the optimization problem associated with x^/q), and convert back to a solu- 
tion of the optimization problem based on x(t Q ). The reason why this is not 
the case, and why a restriction such as (12.2-7) has to apply for all t and not 
even just for t Q , is as follows. Equation (12.2-15) requires that in passing from 
x(-) and «(•) to Xj(-) and u x (»), the relation x t (/) + Guf(t) = x(t) should 
hold for all t. 

This equation has two interpretations, one for t = t Q and one for t > t 0 . 
For the case of / = it implies that the optimization problem of minimizing 
(12.2-8) with respect to (12.2-7) for arbitrary x { ((q) should be subjected to a 
side constraint on the value of the optimal control at time t Q — namely, 
*i(*o) 1 " (fo) x (to)> tn order that its solution may be used to provide 

information concerning the original optimization problem. Without this con- 
straint, we know that the optimum procedure is to set uf(f 0 ) = K'(t 0 )x x (t 0 ). 
Therefore, the modified optimization problem only yields a solution to the 
original optimization problem if, given x(^ 0 ), x,(t Q ) can be chosen so that 

*iOo) G(t 0 )K (^o)-^i(^o) = *Oo)- 

In view of the result K'G = —I, evidently a necessary condition on x(/ 0 ) 
is that K'(t 0 )x(t 0 ) = 0. This is also a sufficient condition on x(/ 0 ), for if x(/ 0 ) 
satisfies K'(t 0 )x(t 0 ) ~ 0, we can define x^/q) = x(f 0 ). Then wf(/ 0 ) = 
K'(t Q )x,(t») = K\tMQ = and therefore the side constraint that 
*i(f 0 ) + GQo MOo) = x ( { o) holds. 

We now interpret the constraint equation x,(t) + Gu x (t) = x{t) for / > / 0 
assuming it holds for t = t 0 . If u*(t) is the optimal control for the modified 
optimization problem, if (12.2-4) is used to set u{t) = u*(t) as the control for 
the original problem (without consideration of its optimality or nonopti- 
mality), and if x = Fx + Gu is used to determine x(t) 7 then it is also neces- 
sary for x(t) to equal x,(f) + Gu x (t). To see that this is indeed the case, 
observe that 

jt[x(/) — x x (t) — G(t)u^(t)] = x — x y — Gu* — Gu^ 

= Fx + Gu * - Fx x - G x u x - Gu* - Gu, 

= F[x — x, — Git,]. 

But since at t = r 0 , x(/ 0 ) - x,(/ 0 ) - G(t 0 )u*(t 0 ) = 0, we have x(/) - x x (t) 
- G{t)u* x {t) = 0 for all /. 

Consequently, K'(t 0 )x(t 0 ) = 0 is a necessary and sufficient condition for 
the optimal control w?(r) of the modified problem to satisfy the constraint 
equation (12.2-15) for all t > r 0 ; this equation is, of course, necessary to draw 
conclusions concerning the first optimization problem for the second optimi- 
zation problem. Note that (12.2-15) implies that K'{t)x(t) = 0 for all t . 
Therefore, ^'(^o) x ( r o) = 0 implies that K\t)x{t) — 0 for all t< 




Sec . 12.2 



LINEAR MODE SOLUTIONS 287 



We now return to the mainstream of the argument. Still considering the 
case when the control u v is optimal, i.e., when u x = uf = K'x 19 we denote the 
corresponding control u as u*, without claiming for the moment that u* is 
optimal. From (12,2-4) and (12.2-9), we have that 

u* = ii* = (K*xl) = K'x x + K'x x . 

From the original system equation and the constraint equation (12.2-15), 
there follows 

w* - K\x - Guf) + K'[F(x - Guf) + (FG - G)u*] 

- K'x + K'Fx - ( K'G + K'G)u * 

- ( K'F- K')x (12.2-18) 

where the final equality follows by use of (12.2-14). 

We now claim that on applying the preceding control, viz., w* = 
(K'F + K')x to the original system, provided that the initial state x(t 0 ) is such 
that K'x(t 0 ) = 0, the performance index \ T to ( x'Qx ) dt is minimized and has the 
value V* = x'(to)P(t Q )x(tQ ). To see that this is so, we shall interpret the stan- 
dard regulator results and the various relationships just developed. We have 
from the regulator results that applying the control uf = K'x , to the system, 
x { = /jfj + G l u l minimizes the index (12.2-8). Equivalent to this minimiza- 
tion process for the case when x(t Q ) is chosen such that K'(t 0 )x(t 0 ) = 0 is 
the application of the control u* = ( K'F + K')xt o the system x = Fx + <7tf. 
To minimize the same index (12.2-8), subject to the constraints (12.2-4) and 
(12.2-5), is precisely the same as minimizing the index j7 o (x'Qx) dt, as we 
remarked earlier. Our claim is thereby established. An alternative argument 
that avoids use of the standard regulator theory is requested in Problem 
12 . 2 - 2 . 

Of course, by simply rejecting those solutions of the singular minimization 
problem which violate the constraints (12.2-2) at any time t > t 0 , solutions 
that remain are the solutions to the singular optimal regulator problem posed 
at the beginning of this section , where we included the control magnitude con- 
straint. That is, for the /th input to operate in the singular mode rather than 
in the bang-bang mode, the constraints 

| (K'Fx + K’xW < 1 and (K'x) i == 0 

must hold at all times. 

We now recall the relay theory of Chapter 6, Sec. 6.2, to see that, provided 
the /th entry uf(t) of the control law u* satisfies |w i *(/)| < 1 for all t > / 0 , 
the singular control law ith component may be realized by a relay in its chat- 
tering mode where the input to the relay is the ith entry of K'x, written as (K'x) i . 
In other words. 



uf — sgn ( K'x) r 



(12.2-19) 
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As explained in the last section, (12.2-19) also serves as a reasonable 
suboptima l control away from the singular strip. Properties of the system 
resulting from using (12.2-19) will be discussed further in the next section. 

The finite-time results just given can be readily extended to the limiting 
case as T approaches infinity, so long as we can establish the existence of the 
limit 

F- lim P(t, T). (12.2-20) 

T — ►c * 0 

Simply because both terms in the integrand of (12.2-8) are nonnegative, it 
follows that x'(t)P(t, T)x(t) increases monotonically as T increases; all that 
is required to establish the existence of P is to show that x’(t)P(t, T)x(t) has 
an upper bound for an arbitrary *(/), which is independent of T. For the 
case when [F, G,] is completely controllable, the existence of the required 
upper bound is established by a direct application of the standard regulator 
results of Chapter 3, Sec. 3.1, but for our purposes, this condition is too 
restrictive. 

We now claim that the condition that [F, G] be completely controllable 
guarantees the existence of an upper bound on x'(t)P(t, T)x(t ), and thus that 
the limit P given by (12.2-20) exists. To see this, recall the existence of a control 
u c9 bounded but not necessarily by unity, which takes the system x = Fx 
+ Gu from an initial state to the zero state in a finite time T l and is zero 
itself after T x . Existence follows because [F, G] is completely controllable. 
Clearly, by using the control w c , we see that the index V(x(t ), w c , /) = 
jj (x'Qx) dt is bounded above for all F> t by some F(||jc(f)||). Now, Prob- 
lem 12.2-2 establishes the following formula, valid for all T, u(*) and x(t): 

j T (x'Qx) dt = x\t)P(t, T)x(t) + [ 7 (x'KK’x) dt. 

This implies that 

x'(t)P(t, T)x(t) < V{x{t\ u e9 1) < V < co 

with F, of course, independent of T. If we let T approach infinity, it follows 
that 

x\t)P{t)x{t) < F(||x(0)|). 

Since x(r) is arbitrary, the desired result follows. The linear mode solutions 
to the singular optimal control problem are now summarized. 

Linear mode solutions to the singular optimal regulator problem . Suppose 
we are given the completely controllable system 

x = Fx + Gu (12.2-1) 

and the performance index 

V(x :(t 0 ), u(-), /„) = f (x'Qx) dt 



(12.2-3) 
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for some nonnegative definite Q and some finite T. Suppose, also, that 
each input u t is constrained by the inequality 

KI<1. (12.2-2) 

Then the optimal control for the ith input of this system may be realized 
by a bang-bang control law (not given here) until the state trajectory 
reaches the singular strip, which is a subset of the singular bounded 
hyperplane 

(K'x); = 0 | (K'Fx + K'x), | < 1 (12.2-21) 

where K' is given from (12.2-1 1) and (12.2-6) as 

K f = -{G f QG)~ x [G\FP + Q) - G'P] (12.2-22) 

and P is the solution of the Riccati equation (12.2-12), which may be 
written 

-P = PF + F'P- KG'QGK' + Q P(T, T) = 0. (12.2-23) 

Once the state trajectory is in the singular strip, it will remain there. 
(Recall that it is this property that defines the singular strip.) The control 
law w* for the singular mode control may be realized by a controller 
gain ( K'F + ^') z - between the system states and input. An alternative 
realization is a controller gain (K') i followed by a relay that will operate 
in its chattering mode. The minimum value of the index jJ o (x'Qx)dt 
for the case when x(^ 0 ) is in the singular strip is 

V*(x(t 0 ), t 0 ) = x'(t 0 )P(t 0 , T)x(t 0 ). 

The results for the limiting case as T approaches infinity are the same as 
before, with P(t, T) replaced by 

P(t) = lim P(t, T). (12.2-20) 

T — 

Figure 12.2-1 shows the two alternative realizations of the singular mode 
control for the single-input case. Lower-case letters are used to indicate vec- 
tors. 



Problem 12.2-1. Given the completely controllable system x = Fx + gu 
and index V ~ ( x'Qx)dt , with Q nonnegative definite, suppose that g'Qg = 0 

and ( Fg — g)'Q(Fg ~ g) ^ 0 . Find a control u* to minimize the index. [Hint: 
Apply in succession the two transformations 

x t = x — gu { Ui ~ u 

and 

*2 = *1 — (Fg — g)u 2 u 2 = Ml 

to convert the problem to a standard regulator problem.] See reference [3] for even 
more general results. 
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(b) 




Fig. 12.2-1 Singular mode optimal control. 

Problem 12.2-2. Show that the system equation x = Fx + Gu and Eqs. 
(12.2-11) through (12.2-13) imply the result 

T (x'Qx) dt - x'(to)P(t 0 , T)x(t 0 ) + T (x'KK'x) dt 

j to J t a 

independently of the control u . Use this result to verify the linear mode solutions to 
the singular regulator problem. 



Problem 12.2-3. For the system 
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find the control that approximately minimizes the index 

V — f (x\ + x\ + * 3 ) dt 
J 0 

subject to |w| < 1. Observe that the Riccati equation is not difficult to solve when, the 
result P [0 0 1 ]' = 0 is used. 



12.3 PROPERTIES OF THE DUAL-MODE OPTIMAL 
REGULATOR 



Throughout this book, we have adopted the philosophy that systems 
for which a quadratic-type performance index is minimized are of interest not 
so much because the index is at its minimum value but because such systems 
have ancillary properties desirable from the engineering point of view. 
Some of the properties of interest discussed in the previous chapters are degree 
of stability, gain margin, phase margin, system sensitivity to parameter varia- 
tions, the ability to accommodate offset errors without affecting the equilib- 
rium point (or desired operating point), and the ability to tolerate time delays 
and nonlinearities in the feedback loop while maintaining a prescribed degree 
of stability. Of course, optimality itself may be of direct engineering signif- 
icance, as in the case of a system whose output is required to follow the 
output of a model while minimizing an error-squared type of performance 
index. 

We now consider the “engineering” properties of systems derived by 
using the singular optimal control theory of the previous section. For 
simplicity, we consider single-input, time-invariant systems and performance 
indices measured over an infinite interval. That is, we consider the system 

x = Fx + gu (12.3-1) 

and performance index 

/• 00 

y( x 0o)> u (‘)> ( o) = (x'Qx)dt (12.3-2) 

J t 0 

where it is assumed that Q is nonnegative definite and that g'Qg is positive. 
Given these assumptions, we can, without loss of generality, normalize Q 
such that g'Qg = 1. The constraint inequality is simply 

I w| < 1. (12.3-3) 

We recall from the previous section that the optimal control is bang- 
bang (i.e., u = + 1 or u = — 1 ) over the entire state space, except for the singu- 
lar strip, which is a subset of the singular bounded hyperplane defined by 

k'x — 0 \k'Fx\ < 1. ( 12 . 3 - 4 ) 
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Here, 

k! = ~g\F'P + Q) (12.3-5) 

and 

P = lim P(t, T) = lim P(t , T) (12.3-6) 

> — oo 

with P( • , T) the solution of 

~P - PF + F'P - (G + PF)gg\FP + Q) + Q P(T, T ) - 0. 

(12.3-7) 

On the singular strip, the optimal control law is linear and is given by 

u* = k’Fx (12.3-8) 

or by 

n* - k\F + yl)x (12.3-9) 

for any y (since k'x = 0 on the singular strip). This law may be realized by 
a linear controller with gain k\F + yl) for some y. An alternative formula- 
tion for w* on the singular strip is 

u* = sgn (k'x). (12.3-10) 

The controller for this case is a linear controller of gain k' followed by a relay 
in its chattering mode. 

To realize the optimal control law off the singular strip is difficult and 
may require an on-line computer. For this reason, we prefer to use one of the 
following realizations, not necessarily optimal, of the closed-loop system. 
In the first realization, the control law (12.3-10) is used for the entire state 
space. That is, the closed-loop system is 

x = Fx + g sgn (k'x). (12.3-1 1) 

A second realization is to use system (12.3-11) until chattering commences 
and then to switch to the control law (12.3-9). This law may be used even if 
disturbed off the singular strip until, say, | k'(F + yJ)x | = 1 or \k’x | = 8 
for some positive 8 chosen by the engineer. When either of these conditions 
is reached, the relay is switched back in the loop. (Recall the discussion in 
Sec. 12.1.) That is, we have a dual-mode control, where the closed-loop 
systems for the two modes are, respectively, 

x = Fx + g sgn (k'x) (12.3- 12a) 

and 

* = [ F + gk'(F -f yi)]x. (12.3-12b) 

Since the system (12. 3-12b) will, in practice, be used off the singular 
strip, the value of y should be chosen carefully. One result given in Chapter 6, 
Sec. 6.2, which is relevant in selecting y, is that off the singular hyperplane 
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k'x = 0 the trajectories approach the hyperplane with a degree of stability 
y. [To see this result, premultiply (12.3-12b) by k' and observe that this reduces 
to 

£{k’x) = -y(k'x) 

where we have used the relationship k'g = — 1 .] With y = 0, trajectories 
off the hyperplane k’x — 0 will never approach the hyperplane, and the sys- 
tem (12.3-1 2b) is therefore not asymptotically stable. 

We now discuss properties of the two dual-mode systems (12.3-11) and 
(12.3-12). The first property of interest is stability, a key aspect of which is 
the stability of both systems when k’x — 0 or, more precisely, when the trajec- 
tories are confined to the singular strip. 

Stability on the singular strip. For the standard regulator problem, an 
observability condition is imposed as a sufficient condition for asymptotic 
stability of the closed-loop system. In the previous section, the minimization 
problem defined by (12.3-1) through (12.3-3) is transformed into a standard 
regulator problem in order to apply the known theory. In particular, the 
transformations 

u x — u x x = x — gu x (12.3-13) 

and definitions 

h=Qg gi = Fg g'Qg = 1 (12.3-14) 

applied to (12.3-1) through (12.3-3) give the problem of minimizing the index 

/•o o 

K(x,(/ 0 ), «,(/„), /„) = [x\(Q — hh')x i + (m, + A'*,) 2 ] dt (12.3-15) 

J to 

subject to the constraints 

x l =Fx l +g l u i |w,|<l. (12.3-16) 

For the case when the optimal control u* for this problem without the 
constraint |u, | < 1 still satisfies this inequality, it is given by 

uf = k'x 1 (12.3-17) 

with k’ defined in the manner stated earlier. A sufficient condition for the 
closed-loop system = (F + £,&')*, to be asymptotically stable when the 
optimal control given by (12.3-17) is used is that the pair [F — Z>J be 
completely observable for any 7), such that D^D\ = (Q — hh’). This is given 
from the regulator theory of Chapter 3, Sec. 3.3, applied directly to the 
preceding problem. With the closed-loop system asymptotically stable, 
u * , given by (12.3-17), and x t decay asymptotically. 

In the previous section, the transformations (12.3-13) are shown to be 
applicable for interpreting the optimal results for the preceding standard 
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regulator problem as results for the original minimization problem on the 
singular strip. Since the transformations (12.3-13) apply on the singular strip, 
a sufficient condition for x = x 1 — gu x to decay asymptotically is that x t 
and u x decay asymptotically. Expressing this condition in terms of the vectors 
and matrices of the original problem definition enables us to conclude that 
on the singular strip the system (12.3-1) with the control (12.3-12) applied is 
asymptotically stable with the following assumption. 

Assumption 12.3-1. The pair [F(/ — gg'Q), D]] is completely observable for 
any D x such that D X D\ = Q(I — gg'Q). 

That is, with Assumption 12.3-1 holding, the closed-loop system 

i - (/ + gk')Fx (12.3-18) 

is asymptotically stable. If x(t 0 ) satisfies 

k'x(t 0 ) = 0 | k'Fx(t 0 ) | < 1 (12.3-19) 

and if the condition \k'Fx(t) | < 1 remains satisfied for all /, both (12.3-11) 
and (12.3-12) are asymptotically stable. An alternative and simpler assump- 
tion now follows. 



Assumption 12.3-2. The pair [F CLf D 2 ] is completely observable for any Z ) 2 
such that D 2 D' 2 = Q , where F C l is the closed-loop system matrix (/ + gk')F . 



To see that this assumption is a sufficient condition for asymptotic stability 
on the singular strip, consider as a tentative Lyapunov function 

V x = x'Fx. (12.3-20) 



Then 



V x = 2x’Px 

= 2 x'FFx + 2x'Pgu 

= -*'(£? - kk')x. (12.3-21) 

The last equality is derived by using the result that Pg = 0 and the rela- 
tionship PF + F'P — kk f + Q = 0. We observe that (12.3-21) holds irre- 
spective of the control u. With our attention restricted to trajectories on the 
singular strip (i.e., k'x = 0), we note that the function V x is, in fact, the optimal 
performance index V*{x{t ), /), for which 



poo 

V* = | (x'Qx)dt. (12.3-22) 

Furthermore, on the singular strip, k'x = 0; therefore, V reduces to 

V* = -x'Qx. (12.3-23) 

Clearly, if Q is positive definite, then V* is a Lyapunov function, and trajec- 
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tories on the singular strip are asymptotically stable. However, in general, 
Q is but nonnegative definite, and thus V * and — K* are only guaranteed to 
be nonnegative. For V* to be a Lyapunov function on the singular strip, we 
require first that V* be strictly positive, and second, that K* be not only 
nonpositive but also not identically zero on any system trajectory, confined, 
of course, to the singular strip (see the stability theorems of Chapter 3, Sec. 
3.2). We now claim that Assumption 12.3-2 guarantees these results for the 
case when x(*) is the solution of (12.3-18) for some x(/ 0 ) satisfying (12.3-19). 
The reasoning is similar to that used in Chapter 3, Sec. 3.2, for the standard 
regulator stability results, and thus is only given in outline form here. 

To see that V* is positive on the singular strip with Assumption 12.3-2 
holding, observe that if K* were zero for some x(/ 0 ), then 

0= x'Qxdt ^l x'(t Q ) exp [F' CL (t — t 0 )]D 2 D' 2 

J fo J to 

X exp [F CL (t — /oM'o) 

This equation implies that D\ exp [F CL {t — f 0 )]x(? 0 ) = O.for a nonzero x(t 0 ), 
and thus it contradicts Assumption 12.3-2. 

To see that V* is not identically zero along a system trajectory in the 
singular strip, suppose initially that it is identically zero. Then x Qx is iden- 
tically zero, and thus V* is identically zero, contradicting the previous result 
when Assumption 12.3-2 is in force. 

The claim that Assumption 12.3-2 is a sufficient condition for stability 
on the singular strip has now been established. This means that under Assump- 
tion 12.3-2 the closed-loop system (12.3-11), when in its singular strip, is 
asymptotically stable. In fact, consider the introduction of nonlinearities p 
into the feedback loop as follows: 

x = Fx + g/?[sgn ( k'x ), t\ (12.3-24) 

where p is a possibly time-varying nonlinearity, with the restrictions that 
P( 1, 0 and P(— 1, t) are bounded, 

fid 0 > fit > 0; fid 1, t)<p 2 < 0 (12.3-25) 

for all t , where fi x and — /? 2 are positive constants, then chattering still occurs 
on the singular strip in such a way as to leave the closed-loop system (12.3-18) 
unaltered. The stability results are thus unaffected (see Chapter 6, Sec. 6.2). 

We also have that for the dual-mode system (12.3-12), once chattering 
of the relay commences for the system (1 2.3-1 2a) and the linear law is switched 
in resulting in closed-loop system equations (12.3-12b), then the system 
will behave in an asymptotically stable fashion, so long as the mode defined 
by (12.3-12b) continues to be the operating mode. However, as pointed out 
earlier, if there is a disturbance from these trajectories and y = 0, then the 
system will not be asymptotically stable. 
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Problem 12.3-1 is concerned with establishing conditions that achieve 
a prescribed degree of stability a, say, on the singular strip. The problem of 
stability off the singular strip is now discussed. 

Stability on and off the singular strip. In Chapter 6, Sec. 6.2, general 
results for relay systems are quoted. One of these results is that, provided the 
trajectories on the singular strip are asymptotically stable, then trajectories 
in the vicinity of the state-space origin are asymptotically stable. That is, 
local asymptotic stability is assured. The proof of the general result is not 
given in Chapter 6, but since, for the particular application of the closed- 
loop system 

x — Fx + g sgn ( k'x ) (12.3-11) 

the result is readily proved, we now do so. 

We consider the function 

V 2 - x'Px + (k'x) sgn (k'x) (12.3-26) 

as a tentative Lyapunov function. Clearly, for the case when x is on the sin- 
gular strip k'x = 0, then V 2 = V x — V*, and, therefore, V 2 is positive defi- 
nite. For the case k'x ^ 0, V 2 will be greater than or equal to \k'x\. Differ- 
entiating (12.3-26) yields 

V 2 = — x'(Q — kk')x + k'[Fx + g sgn (k'x)] sgn (k'x) 

= — x'(Q ~ kk')x + k'Fx sgn (k'x) — 1. (12.3-27) 

For the second equality, the result k'g = —1 has been used. Observe that 
for sufficiently small x the term V 2 is negative. This means that V 2 is a Lya- 
punov function for a sufficiently small region in the vicinity of the state- 
space origin. The local asymptotic stability result is thereby established. 

It is also readily shown that the system given by (12.3-24) and (12.3-25) 
is locally asymptotically stable. That is, the introduction of nonlinearities 
of the form (12.3-25) into the system input transducers does not affect the 
preceding stability result. 

We now ask if anything can be said about the global asymptotic stability 
of the system 

x = Fx + g sgn (k'x). (12.3-1 1) 

One result is that (1) if the system x = Fx is asymptotically stable, and (2) 
if the real part of (l +jqco)k'(jcoI~ F)~ l g is positive for alt co and some 
nonnegative q, then the closed-loop system (12.3-1 1) is globally asymptotically 
stable. This stability result is discussed in Chapter 6, Sec. 6.2. Unfortunately, 
it does not help much here except in particular cases. 

There is another result that could be mentioned, which is relevant to 
the problem. We claim that if the system x = Fx is asymptotically stable , 
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then the system that minimizes the performance index J~ (x'Qx) dt will be 
globally asymptotically stable , assuming that the singular strip is asymptotically 
stable. (We have already established that Assumption 12.3-2 guarantees the 
singular strip to be asymptotically stable.) To see that this result is so, we 
observe that the performance index is finite for a zero control and therefore 
must be finite for the optimal control. But from (12.3-20) and (12.3-21) 
(which are independent of the control w), we have 

poo poo 

(x'Qx) dt = x'Px + ( x'kk'x ) dt. 

This means that with {x'Qx) dt finite, then either k'x = 0 or k'x approaches 
zero as t approaches infinity. However, the only way for k'x to be zero or 
approximately zero for any length of time is for the trajectories to be in the 
vicinity of the singular strip. An examination of the function V 2 [see (12.3-26)] 
and its derivative [see (12.3-27)] shows that when Assumption 12.3-2 holds 
and when the trajectories are in the vicinity of the singular strip, V 2 is a 
Lyapunov function. Our claim is now established. Note again that this result 
is true in general only when the truly optimal switching surface is used, rather 
than the (suboptimal) hyperplane k'x = 0. 

To establish global stability results for the dual-mode system (12.3-12) 
appears just as difficult as for the relay system (12.3-11). But for the linear 
mode of operation , it is possible for the linear controller to be the same as that 
for a standard regulator. This means that the linear mode of operation can 
have all the desirable properties associated with the regulator. This we now 
investigate. 

Linear mode design for dual mode control. We present two approaches 
to the design of a linear mode controller for a dual-mode system. Both achieve 
a control law that is identical to one achieved using standard regulator theory, 
and both are based on the theory of the previous section. The idea of achiev- 
ing the same controller as for a standard regulator is simply to guarantee the 
desirable properties associated with these regulators, such as a prescribed 
degree of stability, good phase margin and gain margin, etc. 

The first approach requires a selection of the constant y in the control 
u = k'(F + yl)x. For any y, the control will be optimal for the singular 
problem on the singular strip, whereas we can argue that for a suitably large 
y, the control will also be optimal for a particular standard regulator mini- 
mization problem. 

We know that for states on the singular strip, (provided Assumption 
12.3-2 holds), the closed-loop system 

X = [F + gk\F + yl))x (12.3-12b) 

is asymptotically stable. Moreover, the degree of stability orthogonal to the 
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singular strip is y. In other words, [F + gk\F 4 y/)] has (n — 1) eigenvalues 
in the left half-plane and one eigenvalue at s = — y. For y sufficiently large, 
the eigenvalues will be the same as that for some standard optimal regulator 
design, as shown in Chapter 7, Sec. 7.4. Problem 12.3-2 asks for further details 
on this result. 

An alternative approach to proving the same result is to consider as a 
tentative Lyapunov function for (12.3- 12b) with input nonlinearities the 
following function: 

V 3 - x f [2 F'PF 4 QF 4 F'Q + y(Q 4 kk’ + 2yP)]x. (12.3-28) 

Its derivative can be determined as 

K 3 - u\ 1 - 2 4) - x'[F\2yP - Q)F + fQ]x (12.3-29) 

where <f> denotes nonlinearities in the sector [-£, oo) at the plant input trans- 
ducer. For some suitably large y, the function V 3 is, by inspection, a Lya- 
punov function (except for certain special cases). Because this is so, the theory 
on the inverse problem of optimal control, given in Chapter 7, yields that 
with a suitably large y, the closed-loop system (12.3- 12b) is optimal with 
respect to a standard quadratic performance index. Problem 12.3-3 asks that 
the details of this approach be worked out. 

Problem 12.3-1. For the linear system x = Fx -f gu, |w|< 1, discuss the 
minimization of quadratic performance indices that achieve a prescribed degree of 
stability. 

Problem 12.3-2. Using the terminology of this section, show that the matrix 
lF + gk'(F 4 y/)] has (n — 1) eigenvalues in the left half-plane and one eigenvalue 
at s = — y. What test could be applied to see if a particular y is sufficiently large for 
the closed loop x = [F 4- gk'(F 4 yl)\x to be optimal in the sense of Sec. 7.3? 

Problem 1 2.3-3. Standard regulator theory yields an optimal control of the 
form u* = —g'P ix, where the term x'P^x is a Lyapunov function for the system 
x = (F — g(f>g'Pi)x . (Recall that 0 denotes a time-varying gain in the sector [£, oo].) 
The optimal control for the singular problem is u* = —g'(F'P + Q){F + yl)x. 
This suggests that we may be able to construct a Lyapunov function using the qua- 
dratic term x'(F'P + Q)(F + yl)x, associated with the system 

x = [F- g<f>g'(F'P + Q)(F + yl)]x 

for some y > 0. Observe also that since g'P = 0, the optimal control u* could be 
written as u* = g[(F'P + Q)(F + yl) + \pp] x for some /J>0. All this suggests 
that we consider 

V = x'[2(F'P 4 Q)(F 4 yl) 4 pP]x 

as a tentative Lyapunov function. Investigate this case, using relationships such as 
PF 4 F'P — kk' 4 Q = 0 and g'P = 0 to obtain for the case fi = 2y 2 the equa- 
tions (12.3-28) and (12.3-29). 
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12.4 OPTIMAL REGULATOR WITH BOUNDS ON THE 
CONTROL DERIVATIVE 



In this section, the theory of the previous sections is applied to solve the 
problem of controlling the completely controllable system 

x^Fx+gu (12.4-1) 

subject to the constraint 

|ii|< 1. (12.4-2) 

The performance index of interest is the usual regulator index 



V(x{t 0 ), u(-),t 0 ) = f ( x'Qx + u 1 ) dt 

J t 0 



(12.4-3) 



where Q is nonnegative definite. We shall consider single-input, time- 
invariant plants, for simplicity. The physical significance of this problem is 
related to the regulator problem with derivative constraints of Chapter 10. 

Clearly, the preceding minimization problem will be of interest for situa- 
tions where rapid changes in the control input are to be avoided. It might be 
thought that making Q sufficiently small in the index (12.4-3) would ensure 
that there would be a small control, and therefore small rates of change of 
the control. In some situations, this may be true (although there is no imme- 
diate theoretical justification for it), but to choose an appropriate Q would 
depend on trial-and-error techniques, and result would probably be too 
conservative. With the constraint (12.4-2), the engineering limits are imme- 
diately taken into account in the design. The only disadvantage is an increase 
in controller complexity, as will become clear in the following development. 

Using the same ideas as in Chapter 10, we define new variables 



« 

u = u 

These definitions lead to 




♦ ^ 

-A r - '' -A I 

x — Fx + gu 

where 



(12.4-4) 

(12.4-5) 




(12.4-6) 



[Equation (12.4-5) represents a system consisting of the system of (12.4-1) 
augmented by an integrator at the input.] The performance index (12.4-3) 
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where 




The control variable constraint is now 



(12.4-8) 



|w| < 1. (12.4-9) 

Application of the theory of the previous sections suggests that an 
effective suboptimal control is 

u - sgn (k'x). (12.4-10) 

Here, 

k' - -(g'QgY'gXFP + Q) (12.4-11) 

and P(', T) is the solution of the Riccati equation, written using (12.4-11) as 

-p - PF+ F'P ~ HgQgY'k' + Q P(T , T ) - 0. (12.4-12) 

The result for the augmented system (12.4-5) can now be interpreted 
as a result applying to the original system (12.4-1). Let us partition P as 




using the same partitioning scheme as for For Q. Substituting all partitioned 
matrices into the equations (12.4-12) and (12.4-11) yields 

-Px - PxF + F'P x - P lg g'P , + Q (12.4-13) 

P 2 -0 

P 3 = 0 

k' = —g'Pi- 

Now, the control u> or, equivalently, w, is given as 

u = u = sgn (k'x) = sgn (~g'P x x — u). 

This means that the differential equation for u is 

u = sgn (— g'PyX — u) (12.4-14) 

where P x is the solution of (12.4-13). This control scheme is shown in Fig. 
12.4-1. 

When the control u is on the singular strip k'x = 0, then we have that 
-g’PiX — u = 0, or 



« = g'P \X* (12.4-15) 

This is precisely the control law that results from the solution to the standard 
regulator problem of minimizing the index (12.4-3) for the original system 
(32.4-1). This result is expected, since the singular strip is that region where 
the control lies strictly within its constraints. 
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Fig. 12.4-1 Optimal regulator with |w| < 1. 

Further investigations of this result are left to the student in the problems. 

Problem 1 2.4-1 . Without verification, give a controller that could be used for 
the system (12.4-1) and index (12.4-3), where the control u must satisfy the con- 
straint |«| < 1. 

Problem 1 2.4-2. Can you think of a scheme which ensures that the constraints 
| u | < 1 and 1 u [ < 1 are satisfied? 

Problem 12.4-3. Discuss how the system of Fig. 12.4-1 responds to a step 
input. Assume that the initial states of the plant and controller are zero. Redraw 
the system where it is required that the output track the input. 
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CHAPTER 



OPTIMAL LINEAR REGULATORS 

WITH CONTROLLER 
CONSTRAINTS 



13.1 FORMULATION OF SPECIFIC OPTIMAL REGULATOR 
PROBLEMS 



Virtually all the regulator results presented so far have been based on 
the assumptions that the system states, or estimates of these, are available 
for use in a control law and that there are no constraints other than linearity 
on the controller. In practice, however, further restrictions on the controller 
may be necessary. Therefore, in this chapter we develop some optimal linear 
regulator results with controller constraints. We refer to these problems as 
specific optimal regulator problems , nonstandard regulator problems , or sub- 
optimal regulator problems. 

Some of the results of this chapter first appeared in references [1] through 
[3]. The remainder of the results and some of the proofs of earlier results 
are new and have not, at the time of the writing of the text, appeared in the 
literature. 

As in previous chapters, we assume we are given an / 7 -dimensional linear 
system 



x(t) = F(t)x(t) + G(t)u(t) 
y(t) - HXt)x(t) 



303 



(13.1-1) 

(13.1-2) 
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where F(-), G(-), and //(•) possess continuous entries. The simplest form of 
control law that we can consider is the nondynamic law 

u(t) - K'(t)x(t). (13.1-3) 

Possible constraints on K(-) [other than the obvious one that K(-) be of 
appropriate dimension] that might be imposed in practice can be classified 
as follows; 

1. Equality constraints on the entries of the controller gain matrix K(*), 
as, e.g., when only the outputs y are available for feedback. That is, 
state estimation is not permitted and the control u has the form 

u — K' Q y — K f Q H'x. 

Equivalently, K is constrained to have the form 

K — NK 0 

for some K 0 of appropriate dimension. 

2. Inequality constraints on the entries of the controller gain matrix, 
as, e.g., when individual gain matrix elements k must satisfy 

k { < k < k z 

for some constants k x and k 2 . The reason for these constraints may 
be physical limitations involved in the construction of controllers, 
or the need to prevent saturation at the plant input. 

3. Constraints that the controller gain have some prespecified functional 
form. We consider the special case of constant or piecewise constant 
gains. Clearly such constraints would be imposed for ease of practical 
implementation of the control law. 

Of course, some or all of the constraints listed could be imposed simultane- 
ously, if necessary. 

In this chapter, we shall consider the optimal design of linear systems 
subject to constraints 1 and 3. The inequality constraints 2 can be handled 
(at least when constraints 3 are also imposed) using nonlinear programming 
techniques, such as are surveyed in [4]. These techniques will not be discussed 
since they are outside the scope of this book. 

Also considered in this chapter are problems involving time-invariant 
plants arising when a dynamic controller is permitted between the plant 
output y and input u. We already know that if the dimension of the con- 
troller is not constrained in any way, the best that can be done in such a situa- 
tion is to first construct a state estimator and then apply an optimal state 
feedback law using the state estimate in lieu of the inaccessible states. For 
the case when the controller dimension is constrained to be less than the 
dimension of the standard state estimator (see Chapter 8), then one approach 
is to estimate only some of the system states (or appropriate linear combina- 
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tions of the states) and use these estimates together with the system output in 
a nondynamic control law. The introduction of a (partial) state estimator 
then reduces the dynamic controller design problem to the nondynamic design 
problem. A second approach uses an artificial augmentation of the plant with 
integrators at its input; nondynamic feedback laws are determined for the 
augmented plant, which become dynamic laws for the original plant. 

For the remainder of this section, a performance index will be considered, 
which is a natural extension of the usual quadratic index used in the standard 
regulator theory. With this index, an optimal control problem is formulated 
that incorporates controller constraints and is amenable to solution. 

A performance index for regulator problems with controller constraints. 

As a starting point in our development, we recall the quadratic index asso- 
ciated with the standard regulator problem — viz., 

F(x(r 0 ), w(-), T) = f [u'(t)R(t)u(t) + x'(t)Q(t)x(t)] dt (13.1-4) 

J to 

with R(t ) positive definite symmetric and Q(t) nonnegative definite symmetric 
for all t in the range [/ 0 , T], To rewrite this index in terms of the present prob- 
lem, we note that when control law (13.1-3) is used, the closed-loop system 
equations are 

*(/) - [F(t) + G{t)K\t)]x{t). (13.1-5) 

The transition matrix <!>(/, r) associated with this system is defined from 

®(f, T) = [Fit) + T) 0>(t, t) - I. (13.1-6) 

Since x(0 = <$(/, t 0 )x(t 0 ) and u(t) = K'(t)&(t 7 t 0 )x(t 0 ), the performance index 
(13.1-4) may be written simply as 

V(x(t 0 ), w, T)\(u = K'x) = x'(/ 0 )P(r 0 )x(r 0 ) (13.1-7) 

where 

Pit o) = f ®'k t,)[K(t)R(t)K'(t) + Q(0Mt, t 0 ) dt. (13.1-8) 

J to 

The matrix P(») is easily verified to be the solution of the linear equation 

~P = P(F + GK') + (F + GK'yP + ( KRK ' + Q) (13.1-9) 

with boundary condition P(T) — 0. See Appendix A for an outline discussion 
of such equations and their solution. 

If we now impose constraints on K( : ) and seek to find an optimum K(-) 
using the index (13.1-7) subject to these constraints, it will be generally true 
that for an arbitrary but fixed x(t 0 ), there will be one (or possibly more) 
values of K { •) which will minimize (13.1-7). Unfortunately, the minimizing 
K(») will probably vary from one .v(/ 0 ) to another. However, for practical 
reasons, it is clearly desirable to have just one K ( •) as the optimal feedback 
gain, irrespective of what x(f 0 ) is. One way out of this dilemma is to assume 
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that the n vector x(/ 0 ) is a random variable, uniformly distributed on the 
surface of the /7-dimensional unit sphere, and to seek the particular feedback 
gain K(') minimizing the expected value, over all x(t 0 ), of (13.1-7). That is, 
we define a new performance index 

V(K(>), T ) - £[4oWo)^o)l (13.1-10) 

with the random variable x(t 0 ) satisfying 

= — / (13.1-11) 

n 



where £[•] denotes expected value. (Problem 13.1-1 asks that an interpreta- 
tion be given for the case when £[*(r 0 )x;'(fo)] = {ljn)A for some A = A' 
> 0 .} 

An alternative simpler expression for this index can be determined 
directly from (13.1-10) and (13.1-11) as follows: 



E[x‘Px] = £ />, /£[*/•*/] = 

i, J 





where x ( denotes the ith element of x, p u denotes the element in the /th row 
and yth column of P, and tr[P] denotes the trace of P. Thus, the index 



fXK(.),T) = Ur[P(t 0 )] (13.1-12) 



is the new performance index with control variable A^(*) [in the sense that 
V is a functional of A^(*)]- 

An alternative performance index to that given in (13.1-12) is V = 
^maxt^o)]* This gives the maximum value of x'Px when x varies on the unit 
sphere, and in some situations it could be more usef ul than the index (13.1-12). 
However, we shall not consider this further in this chapter. 

Let us summarize in one sentence the new minimization problem we have 
implicitly posed in the above arguments: Given matrices P(»), (7(-), (?(*)> 
and P(*), with the various restrictions assumed previously, find a matrix 
K(») to minimize the index (13.1-10) with (13.1-1 1) holding, or, equivalently, 
the index (13.1-12), subject to whatever restrictions are imposed on /f(«). 
This nonstandard optimal control problem has sometimes been referred to as 
a suboptimal linear control problem or a specific linear control problem. 

When considering the limiting case as the terminal time T approaches 
infinity (the infinite-time case), we shall assume that the plant is time invariant 
and the control law is time invariant. (The optimal law for such a case may be 
time varying (see [5]).} With these assumptions, the optimal index is 

V(K) = E[x'(t 0 )Px(t 0 )] 



E[x(t Q )x'{t 0 )} = —I 

n 



with 
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or, equivalently, 

V(K) - — tr[F] 
n 

where 

P = r exp [{F + GK')'t]{KRK' + Q) exp [(F + GK’)t] dt . 

The matrix P is the solution of the linear equation 

P(F + GK') +(F+ GK’yP + (KRfC + Q) - 0. 

Techniques for the solution of such equations are discussed in Appendix A. 

For this infinite-time case, the optimum K must, of course, satisfy the 
further constraint that the closed-loop system 

x = (F + GK')x 

he asymptotically stable. (Otherwise, the optimal closed-loop system will be 
of little practical utility. Also, V certainly will be infinite.) This raises the 
question as to whether there exists any constant control laws K (let alone an 
optimal one) that will satisfy the imposed constraints and the asymptotic 
stability constraint. Unfortunately, this existence question has as yet no 
straightforward answer, and the related problem of finding such a AT, assum- 
ing one exists, has also not been solved in any convenient manner. 

Of course, observability of [F, D] for any D such that DD' Q is not 
sufficient to guarantee the desired stability, although this condition does guar- 
antee that P is positive definite, provided that F + GK' has all eigenvalues 
with negative real parts (see Problem 13.1-3). 

Before we consider analytical results associated with minimizing the 
performance index just introduced, it is interesting to observe one immediate 
and simple link between the nonstrandard optimal case and the standard 
optimal case. This is, that if there are no constraints such as listed earlier in this 
section on the controller gain , then the controller gain A(*), which minimizes 
the index (13.1-12) associated with the nonstandard optimal problem , is the 
same gain resulting from the solution of the standard regulator problem with 
index (13.1-4). To see this, let us assume that the optimum value of the index 
(13/1-12) is (1 In) tr[P] and the optimum value of the index (13.1-4) is x'P*x. 
Assume the corresponding control laws are K and A*, and that they are differ- 
ent. Then, because K is not optimal for the index (13.1-4), x'Px>x'P*x for 
all x, with strict inequality for some x. in other words, P - F* is nonnegative 
definite, and nonzero. Hence, tr(P - P*) > 0, and so tr(P) > tr(P*). This 
contradicts the optimality of K for the index (13.1-12). Hence, K =r K* and 
P = P*. 

When the optimal law is constrained, then, in general, K will not be the 
same as AT*, the optimal law associated with the unconstrained problem. It 
appears, then, that the nonnegative quantity trfP] — tr[P*] gives some meas- 
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ure of how “close” a nonstandard optimal regulator is to a standard one. 
This “closeness” concept is an interesting one that will be investigated in 
further detail in later sections. 

Problem 13.1-1. What is the mathematical and practical significance of a 
performance index of the form 

V = xXt 0 )P(t 0 )x(t 0 ) 

where FM/o)*(/o)] is (1 In) A for some positive definite symmetric A matrix — rather 
than the unit matrix? Show that for this case, V ~ tr [AP]. 

Problem 13.1-2. A second approach to the problem of constructing con- 
strained dynamic controllers relies on the augmentation ideas of Chapter 10. Discuss 
how the ideas of Chapter 10 could be used, and present a detailed problem formula- 
tion. 



Problem 13.1-3. Show that if (FT GK) has all eigenvalues with negative 
real parts, and if [F, D] is completely observable for any D with DD' — Q , then the 
solution P of the following equation is positive definite: 

P(F + GK') + (F + GK'yP + (KRK' + 0 = 0. 



13.2 ANALYTICAL RESULTS AND COMPUTATIONAL 
ASPECTS 



Tn this section, we derive analytical results that are useful in formulating 
algorithms to compute feedback gains for linear regulators with controller 
constraints. The algorithms themselves are not presented here except for the 
simplest and one of the most useful cases. For the other cases, the reader 
is referred to references [1] through [3]. Throughout the section, we consider 
cases arising when controllers are constrained to be nondynamic. However, 
as a later section shows, the results have application to the cases when 
estimates of some of the states (or linear combinations of the states) are avail- 
able. When finite-time optimal control is considered, the matrices are per- 
mitted to be time varying, but when infinite-time results are derived toward 
the end of the section, the matrices are assumed constant. The infinite-time 
results have the most ready application, but there are stability difficulties, 
as will be discussed. 

State feedback controllers with constant or piecewise constant gain ele- 
ments. Let us consider the /7-dimensional closed-loop system 

x(t) = [F0 + G(t)K'(t)]x(t) (13.2-1) 

with K(-) either constant or piecewise constant. We seek to minimize the index 
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P(K(.),T) = — tr[P(/ 0 )] (13.2-2) 

where P(«) is the solution of the linear equation 

-P = P(F + GK') + (F + GKJP + (KRK f + Q) (13.2-3) 
with boundary condition P(T) = 0. That is, 

no - f r a>'(/, + ewm * 0 ) * (13.2-4) 

where <!>(•, •) is the transition matrix associated with (13.2-1): 

®(/, t) - [F(t) + G(t)K'(t)Wt, t) ®(t, t) - /. (13.2-5) 

We shall assume that the interval [> 0 , T] is divided into m fixed intervals (not 
necessarily equal) with K constant in each interval. We denote the value of 
the matrix K in the /th interval /J by K ]. For the moment, we assume 
no further constraints on K. 

Clearly, necessary conditions for any set of to be optimum are that 



d 



trtno] 

dK t 




(13.2-6) 



(See Appendix A for the definition of differentiation of a scalar with respect 
to a matrix.) 

We now seek an expression for d tr [P(/ 0 )]/<?/f, in terms of the and 
known quantities. The notation k ijl will be used to denote the entry in the 
yth row and /th column of the matrix the notation e. to denote a vector 
with all entries zero except the /th entry which is unity, and the notation a t 
will denote a function with value 1 on [t i ^ 1 > fj and zero elsewhere. 

Formally, then, differentiating (2-3) with respect to k , there obtains 

(on equating and dP/dk u , ) 



d_ 

dt 




dP (F + GK’) -1- (F + GK')’ dP 



dkiii 



dk UI 



+ ;■ G'P) 1 ■ a, (KR + PG)e,e' ] . 



From this equation and the boundary condition dP(T)jdk ijl = 0, it follows 
that 



= f V(t, t,)[e } e',{RK' + G'P) f (KR 4- PG)e 1 e' i ]^(t, t Q ) dt. 

UK ijl J ti-i 

Taking the trace of both sides, and using the well-known properties (see 
Appendix A), tr| A\B + B r )A] — 2tr[A'B'A\ = 2 tr [B'AA'), gives the sim- 
plification 



d tr[P(t 0 )] 
dk ul 



f tr [(KR I PG)e,e'<&(t, t 0 )<S>'(t, /„)] dt. 
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This result may be simplified even further by application of the property 
tr| ab'] - b' a to yield 

3 tr jf (?o)] = f ‘ *Mt, t 0 )V(t, t 0 )(KR + PG)e t dt, 

UK ijl J ti-i 

which, in turn, yields an expression for d tr [P(t 0 )]/dKj — namely, 

- tr ^° )] = _ ®(f, * 0 )®'(', t 0 )[K t R(jt) + P(t)G{t)] dt. (13.2-7) 

Thus, the necessary conditions for optimality are that 

f hW{t> QiKiKi) + P(t)G(t)] dt = 0. (13.2-8) 

This same result is derived in [1] via a different procedure. 

Clearly, the necessary conditions and expressions for partial derivatives 
that we have just developed may be incorporated into algorithms for finding 
optimum values for the K t (see [1])- However, as indicated in reference [1], 
the task of algorithm construction and application is a formidable one for 
both programmer and computer; therefore, further discussion of this aspect 
of the problem will not be included here. 

For the case when K is constrained to be piecewise constant and there 
are a large number of time intervals used, the results are, as expected, rea- 
sonably close to the optimum K(-) derived assuming no constraints on K. 
On the other hand, when the number of intervals is small (say, less than five), 
the sequences of optimum K t in some examples considered in [1] bear no 
apparent relation at all to the optimum K{*) with no constraints. 

Output feedback controllers with constant or piecewise constant gains. 

We now ask: What are the necessary conditions for the same problem as just 
treated, with the additional constraint that the system be controlled by memo- 
ryless output feedback rather than memoryless state feedback? For this case, 
the control u has the form u = K f 0 y = K'qH’x for some AT 0 of appropriate 
dimension. That is, 

K = HK 0 . (13.2-9) 

Of course, K 0 (t) = AT 0i -, a constant matrix, on the /th interval. Using (13.2-9) 
and repeating the same arguments as before, yields an expression for 
d tr P[t Q ]/dK Qi : 

= r to W(t, t 0 )[H(t)K 0i R(t) + P(t)G(t)] dt 

ViV 0 / J tt-i 

(13.2-10) 

where, of course, 

~P = P(F + GK' 0 H’) + (F + GK'oHJP 
+ (HK 0 RK' 0 H' + Q) 



(13.2-11) 
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with boundary condition P{T) ~ 0, and 

6(/, t) - [F(0 + t) 0>(t, t) = /. (13.2-12) 

The necessary conditions for optimality are now 

f H'(tWt, t 0 )d>\t, t ,.)[H(t)K ai R(t) + P(t)G{t)} dt = 0 

J fi-t 

i = 1,2,..., m. (13.2-13) 

The computational problems associated with this condition for optimality are 
clearly only minor modifications of the ones where full state feedback is 
allowed. 

Output feedback controllers with time-varying gains. We now relax the 
constancy or piecewise constancy constraint, but insist that K(t) = 
H(t)K Q {t) — i.e., demand output feedback. For this case, necessary conditions 
for optimality may be obtained from the piecewise-constant output-feedback 
case by a limiting operation. We let the time interval over which gains are to 
be constant be no larger than At, and then let At approach zero. Such a 
procedure can be justified by a theorem associated with the Ritz method for 
calculating optimal controls (see [6]). Condition (13.2-13) simply reduces to 
requiring that 

H'm(t, t 0 )V{t, t 0 )[H{t)K 0 (t)R(t) + P(t)G{t)} -- 0 

for all t in the range [ t Q , T]. That is, K 0 must satisfy 

K’ 0 = -R-'G’P&Q'HlH'Wb'H]- 1 (13.2-14) 

where we have used <5 to denote 4>(/, t 0 ). (Problem 13.2-1 asks that the case 
when the inverse of H'&i&'H fails to exist should be considered.) Of course, 
P and O satisfy (13.2-11) and (13.2-12), respectively. These necessary condi- 
tions are also sufficient conditions , as may be seen by forming a Hamiltonian 
and showing that the Hamilton-Jacobi equation is satisfied. This is done in 
reference [2]. (See also Problem 13.2-2.) 

It is interesting to observe that when H = I, corresponding to state- 
variable feedback, then K 0 in (13.2-14) satisfies 

K f 0 - -R' l G'P 

and P is given from 

-P - PF + F'P - PGR-'G'P + Q P(T) = 0. 

In other words, the now familiar standard finite-time regulator results are 
recovered. 

Infinite-time results for output feedback controllers with constant gains. 

Let us consider the time-invariant, ^-dimensional system 

x = Fx + Gu 
y = H'x 
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where the control u is constrained to have the form 

u = K' 0 y = K'qH'x 

for some constant matrix K 0 of appropriate dimension. We seek to minimize 
the performance index 

V(HK 0 ) = — tr[P] 
n 

where P is the solution of the linear equation 

P(F + GK' 0 H') + (F + GK'oHJP + (HK 0 RK f 0 H' + Q) = 0 
provided, of course, that the closed-loop system 

x = (F + GK ! 0 H f )x 

is asymptotically stable. (Otherwise, the index has no physical significance.) 

The necessary conditions for optimality may be determined from the 
limiting case of the finite-time result (13.2-13) as 

//'exp [(F + GK’ 0 H')t] exp [(F + GK' 0 H')’t][HK 0 R + PG] dt = 0 

d 0 

or, more conveniently, as 

H'M(HK 0 R + PG) = 0 

where M is the solution of the linear equation 

M(F + GK' 0 H'y + (F + GK' 0 H')M + 1=0. 

[Of course, these results could be proved directly by using the same approach 
as that in the previous section. This is requested in Problem 13.2-3 — (see also 

[3]).] 

An alternative statement of the necessary condition for optimality is 
that K 0 satisfy 

K' 0 = -R-'G'PMH(H'MH)- 1 . 

[Compare this result with the time-varying result (13.2-14).] Using the nota- 
tion * to indicate optimality, we now rewrite the necessary conditions for 
optimality as follows: 

1. F + GK^'H' has all eigenvalues with negative real parts. 

2. There exist positive definite symmetric matrices P and M such that 
the following three equations hold: 

P(F + GK+H') + (F + GKt'HjP 

+ HK*RK*'H' + Q = 0 

M(F + GK*'H')' +(F + GK* f H')M +1=0 

K*' = —R~ l G' PMH(H'MH) ~ 1 . 

(13.2-15) 
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Several points can now be noted. First, for a given triple F, G, H , there 
may be no K 0 such that F + GK' 0 H f has all eigenvalues with negative real 
parts. (See Problem 13.2-4.) Hence, there may be no optimal control law 
K$. Second, the unknown matrices in (13.2-15) — viz., K%, P, and M — are 
related in a highly nonlinear fashion; it is not possible, in general, to deduce 
from (13.2-15) a quadratic matrix equation for P or M whose solution then 
allows simple computation of the other two matrices. In fact, there is no 
technique for ready solution of (13.2-15). Third, precisely because of the 
nonlinearity of (1 3.2-1 5), it is not known whether there exist other K 0 than the 
optimal AT which, with an associated P and M, satisfy the equations. Con- 
sequently, any iterative technique leading to a solution of (13.2-15) might 
possibly not yield the truly optimal K Fourth, in the special case when 
H 1 exists, (13.2-15) reduces to the standard regulator problem (see Prob- 
lem 13.2-5). This is, of course, as one might expect, since with H l existing, 
the states can be recovered from the outputs. 

We shall now briefly consider procedures for computing K*, P, and M . 
The two following procedures may lead to spurious solutions of (13.2-15) — 

i.e., solutions that do not define the optimal K* ; furthermore, neither proce- 
dure is actually guaranteed to converge. The first procedure, [3], is as follows. 
(Brief remarks follow each step of the procedure.) 

1. Choose K ( 0 0) such that F + GK ( 0 oy H' has all eigenvalues with negative 
real part. [The construction of such a K ( 0 0) appears to be a difficult 
problem in general, for which no explicit procedure is known. Of 
course, if F has all eigenvalues with negative real parts, K ( o 0) = 0 
is satisfactory.] 

2. Define P (n from K%~ 1) as the solution of 

P (n [F + GK ( Q~ iy H'] + [F + GKt l)/ H f YP u) 

+ HK%~ x) RK { i~ xy H' + 2 = 0. 

(This equation is linear and comparatively easy to solve.) 

3. Define M (i) and K ( f from P u) by simultaneous solution of 

M (n [F + GKP’H'Y + [F + GK^'H']M U) +1 = 0 (13.2-16) 

Kf y = -R''G'P u 'M {n H[H'M {n H]‘ x . (13.2-17) 

More precisely, substitute for Ktf in (13.2-16), using (13.2-17). 
This yields a nonlinear equation for A/ ( °; solve this equation, and 
then obtain K ( f from (13.2-16). (There seems to be no quick technique 
for tackling the nonlinear equation.) 

4. Check that [F + GK ( Q iY H'] has all eigenvalues with negative real parts. 
If this is the case, return to (2). (If not, the algorithm fails.) 

5. Take = lim K%\ etc. 

i—*oo 

Although this procedure is not guaranteed to converge, either to the optimal 
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or to any solution of (13.2-15), it does guarantee that tr[P a) ] converges to 
a limit as / — > oo (see [3]). Of course, the fact that tr[P a) ] converges does not 
imply that the P u) converge, as the following example shows: 




2 + (— 1 ) 

0 




However, one suspects that convergence of tr[P (,) ] in some way will “encour- 
age” convergence of P (i) . 

The second computational scheme removes the difficult step of solving 
a nonlinear equation to obtain M (n , which is required in the first computa- 
tional scheme. Only linear equations need be solved. However, the penalty 
paid is that no information regarding convergence is known— not even 
concerning tr[P m ]. As for the first scheme, the algorithm breaks down if 
at each stage a certain matrix fails to have all its eigenvalues with negative 
real parts. An outline of the scheme is as follows: 



1. Choose A^ 0) such that F + GK ( 0 0) H' has all eigenvalues with negative 
real parts. 

2. Define P U) and M (i) from as the solutions of 



P (n [F + GK$- iy H'] + [F + GK“- 



iy H']'P (i) 



+ HKt x) RKt xy H' + Q = 0 



and 



M m [F + GK (i ~ lY H'Y + [F + GK (i ~ iy H']M u) + / - 0. 

3. Define K U) from P u) and M U) as 

K% Y = -R- l G'P in M U) H[H'M i0 HY l . 

4. Check that F + GK ( J y H' has all eigenvalues with negative real parts. 
If this is the case, return to (2). If not, the algorithm fails. 

5. Take K$ — lim K%\ etc. 

/ — >oo 

Other approaches to computation could perhaps be based on gradient 
procedures, with <?[tr P]/dK 0 even being computed experimentally by perturb- 
ing K 0 . 

An example drawn from [3] is the following. The system given has 





h = [0 1] 



and is evidently stabilizable by memoryless output feedback. The Q matrix is 



1 0 “ 
0 0 




Sec . 13.2 



ANALYTICAL RESULTS AND COMPUTATIONAL ASPECTS 315 



whereas R = [1]. This leads to the optimal control law u ~ That is, 

** = “VI, with 

1.4 0.5 
0.5 1 

and the optimal performance index = tr[T] = 2.4. With state feedback, 
the control u = [—0.4 — 0.9]_v is optimal, and the optimum performance 

index is 




*'( 0 ) 



1.3 0.4 
0.4 0.9 



v(0). 



Averaging over all points on the unit sphere, we obtain 2.2 for the averaged 
performance index. 



Problem 1 3.2-1 . Explain how the situation where is singular can be 

dealt with, to deduce an equation akin to (13.2-14). [This also applies to (13.2-15).] 

Problem 13.2-2. Show that the conditions (13.2-12) are sufficient for opti- 
mality when the controller gain may be time varying. {Hint: Use the Hamilton- 
Jacobi theory of Chapter 2. Let X{t) be an n X n matrix satisfying 

X = (F + GK')X X(t 0 ) given. 

We regard the entries of X, arranged as a vector if desired, as the entries of the state 
vector, and the entries of K as the entries of a control vector. As the performance 
index V(X(t ), K(-), t) (the time variable referring now to the initial time), we take 

V{X{t\K{-\t) = tr r X'(t)[K(t)R(t)K'(t) + Q(t)]X(t) <?t, 

j t 

which we hypothesize has minimum value 

V*(X(t\ t ) - tr [X\t)P{t)X{t)] 

for some nonnegative definite matrix P{t ). Verify that dV*jdX(t), which is a matrix 
whose / - j entry is dV/dXijit), is 2 P{t)X{t). The Hamilton-Jacobi equation is 

jr t) = -min {tr [X'(HK 0 RK’ 0 H' + Q)X] 

Ot KM 

+ s (2 PX)ij(FX + CK' 0 H'X),j\ 

i,j 

where the argument of the matrices within { • • • ] has been omitted for clarity. Check 
that this may be rewritten 

-f - V*(X(t\ t) - -min [tr[X '(HK 0 RK f 0 H' + Q + 2 PF + IPGK^H^X]}. 

Ot Ko(i) 

Verify also that 

~ tr[X'(HK 0 RK’ 0 H' + Q + 2 PF + 2PGK' 0 H')X ] 

= 2 (RK' 0 H'XX'H + G'XX'H). 
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Setting this derivative equal to zero allows computation of the minimizing K 0 . 
Verify that when this minimizing K 0 is inserted into the Hamilton-Jacobi equation, 
and the left side of the equation is evaluated, then the differential equation given in 
the text for P(t) is recovered.) 

Problem 1 3.2-3. Derive directly the necessary conditions for optimality (1 3.2- 
15) without specializing the finite-time results derived earlier in the section. How 
w’ould the conditions be changed if the closed-loop system were constrained to have 
a prescribed degree of stability of at least a? What happens when the index is 
(l/w)tr [AP] for some positive definite symmetric A? 

Problem 13.2-4. Show that there exist matrix triples F, G, H y such that 
F -b GK'qH' can never have all eigenvalues with negative real part, irrespective of the 
choice of K 0 . For ease, consider a single-input, single-output system. (Reference 
[5] gives such an example, which has the interesting property that a time-varying 
gain K can be found such that the closed-loop system is stable.) 

Problem 13.2-5. Show that the assumption that H is invertible in the equa- 
tion set (13.2-15) leads to these equations being essentially the standard regulator 
equations. 



13.3 PROPERTIES OF OPTIMAL SOLUTIONS 

Throughout this book, we have stressed the fact that optimality per se 
may not be important, but ancillary properties associated with certain optimal 
solutions may be important. We recall that for the standard optimal regulator 
there are a number of important and pleasing properties associated with 
optimality, and so we are led to ask what are the ancillary advantages of 
using feedback, particularly output feedback, which is optimal in the sense 
of this chapter. First and obviously, the controller constraints we have 
incorporated into the designs normally lead to controllers that are simpler 
to implement physically than a controller design using state-variable feedback, 
as in a standard optimal regulator. Second, the closed-loop system resulting 
from optimal output feedback is asymptotically stable — at least for the 
infinite-time case considered. This is hardly a significant advantage, however; 
in contrast to the state-variable feedback case, one cannot use the optimal 
design procedure as a technique for stabilizing an unstable open-loop system, 
since the first step in the computation of the optimal output feedback law 
generally requires the choice of some output feedback law stabilizing the 
system. 

It appears that it may not be possible to make general statements about 
the properties of the nonstandard optimal systems, in which case it would be 
helpful to have at least some indication of how good the closed-loop systems 
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really are. It might be argued that the index tr[P] gives us such a measure, 
particularly if we have available the value of tr[P*] associated with the cor- 
responding standard optimal regulator. (The closer the two indices, the 
closer we might expect the system properties to be to one another.) 

For the case when an infinite-time optimization has been carried out, 
it is possible to do a little better than the preceding comparison by consider- 
ing the extent of tolerance of nonlinearities in the plant input transducers 
and relating this to what is already known for the standard regulator. This 
eliminates the need to solve any corresponding standard regulator problem. 

Tolerance of nonlinearities in a closed-loop system. We recall that 
arbitrary nonlinearities in the sector [i, oo] can be accommodated in the 
standard optimal regulator without loss of the guaranteed asymptotic stability 
properties. Associated with this property are good sensitivity properties, 
good phase margin, tolerance of time delays, etc. We now show that for any 
nonstandard optimal system, it is possible to calculate a sector [k^k 2 ], 
normally with /c, > k 2 < oo, such that nonlinearities in this sector do not 
disturb system stability. The interval [k x , k 2 ] is then a performance measure 
that can be directly interpreted in physical terms. Moreover, the knowledge 
of [a:,, k 2 ] gives us an indication of how “close” a particular nonstandard 
regulator system is to the standard one — without having to solve the corre- 
sponding standard regulator problem. Actually, the results that will be devel- 
oped are applicable, with modification, to any system which is not a standard 
optimal one, but they are presented here for the case when we have a non- 
standard optimal system design and we desire to evaluate its merits. 

Let us consider the time-invariant plant 

x — Fx ; Gu y -- H'x 

with a nominal control law u K'x. This control law has the form K — HK 0 
for some A' 0 and is optimal in the sense described in the previous section. 
That is, the performance index tr[T] is minimized where P satisfies the linear 
equation 

P(F + GK r ) f (F +- GK'YP + (KRK' + Q) - 0 

for some positive definite symmetric R and nonnegative definite symmetric 
Q. Implicit in what we have assumed is that the closed-loop system 

i - (F f GK')x 

is asymptotically stable. For ease in subsequent calculation, we assume that 
KRK ' + Q is nonsingular, thus guaranteeing that P is positive definite. 

Let us suppose that, in fact, the feedback law is u (f>(K'x, t) where 
the vector 0(«, •) represents nonlinearities (or even time-varying gains) in 
the system input transducers. We now investigate the stability properties of 
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the closed-loop system 

x = Fx + G4>(K'x,t). (13.3-1) 

In particular, we seek sector bounds [jc 15 tc 2 ], such that if 

(k { + e)y'y < 0'(> 7 , t)y < (k 2 - e)y'y (13.3-2) 

for all y and t and some positive e , the closed-loop system (13.3-1) is asympto- 
tically stable. 

For the general multiple-input problem, it appears that explicit formulas 
for k { and k 2 are not available, although and k 2 can be defined implicitly. 
In an effort to obtain explicit results, we shall therefore restrict ourselves 
somewhat. One obvious restriction to impose is to constrain 0(*, • ) to be a 
diagonal matrix, or, equivalently, to decouple any nonlinearities in the 
input transducers. This assumption is reasonable, but unfortunately still leads 
to some difficulty in any attempt to calculate explicit expressions for the bounds 
[k 19 k 2 ]. If, however, we now make the further restriction that <f>(K'x , /) has 
the form k(()K'x for some scalar gain /c(/), whose time-variation is perhaps due 
to nonlinearities , then the bounds [k { ,k 2 \ — now bounds on tc(t ) — maybe 
found explicitly. This restriction is admittedly unrelated to any usual practical 
situation, except, of course, for single-input systems,! when it is no 
restriction at all. However, it does enable a quick computation to be made 
for [/c,, a: 2 ]. 

With the assumption that <f)(K'x , /) is of the form ic(t)K 'x, some scalar 
#(•), we now show that the bounds [ic, , k 2 ] are given explicitly by 

^ 1 +a»in [E])-' (13.3-3) 

= 1 +a ma *[F])- 1 (13.3-4) 

where 

E=[PG K]'(KRK' + Q)'[K PC]. (13.3-5) 

Although the expressions for the bounds k x and k 2 appear to be complicated, 
they are, in fact, not really difficult to calculate once P has been determined 
as part of the nonstandard optimal design. In fact, for the case when we have 
a single-input system (C replaced by g, K replaced by k, etc.), the eigenvalues 
A min [E] and ^ mait [Zs] may be computed to be 

= k\kk' + QY'Pg 

± Ig’Pikk' + QY'Pgk'ikk ' + QY'kV * (13.3-6) 

Problem 13.3-1 asks for verification of this result, given Eq. (13.3-5). 

To investigate the stability properties of the closed-loop system (13.3-1), 

tFor single-input systems, sector bounds [tc u k 2 1 can also be determined by using a 
Nyquist plot together with the Circle Criterion. 
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we adopt as a tentative Lyapunov function 

V — - x'Px. 

Since P is positive definite, it remains for us to examine V. 

Differentiating yields 

V — x'Px + x'Px 

- x'[P(F + K(t)GK') + (F + K(t)GK')'P\x 

- - x'(KRK' + Q)x + (K(t) - 1 )x'[PGK' + KG'P]x. 

The function V is negative definite for all time, provided that the inequality 

(KRK ' + Q) - (k - 1 )[PGK' + KG'P] >el> 0 (133-7) 

holds for all time and some positive constant e. (The notation > [>] is 
shorthand for nonnegative [positive] definite.) The inequality (133-7) thus 
guarantees asymptotic stability of the closed-loop system (133-1). 

We now introduce the identifications 



A ----- ( KRK ' + QY X/2 PG B - (KRK' + Q)~ l 2 K. 



These imply that 




A’B A'A 
B’B B'A 



where E is as defined earlier. The preceding identifications enable us to write 
the inequality (133-7) as 

I-{k- 1 )[AB' + BA'] > 6/ > 0 

which is equivalent to requiring that the nonzero eigenvalues of [AB’ + BA\ 
written X L {AB' + BA\ satisfy 

{k - 1 )X t [AB' + BA'] < (1 - e)l 

for all /. But it may readily be shown that the nonzero eigenvalues of 
[AB' + BA'] are identical to the nonzero eigenvalues of E . (See Problem 
13.3-2.) The preceding inequality is thus equivalent to 

(k- \)Y{E]<( 1 -e)I (13.3-8) 

for all /. For the usual case, when k min [E] and X m!iX [E] are opposite in sign, it 
is immediate that this inequality is equivalent to 

(1 ' F)(X min [E])^ < k ~ 1 < (1 - e)(i max [F])^. 

The bounds on k(() given by (133-3) and (13.3-4) are now immediate. In 
case A min [£] is not negative (A max [£] is not positive), then the bound Xjl/cJ 
ceases to exist. ^ 3 

Problem 133-3 requests the reader to show that when K = —PC', 
corresponding to the standard optimal regulator case, sector bounds of ^ 
and oo result from these calculations. 
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To conclude this section, we give some indication of the results possible 
in the general multiple-input case. Suppose K'x is an in vector. We consider 
a class of $(•, •) defined through an m x m matrix O(r): 

t ) = <J>( t)K'x. (13.3-9) 

[The time variation in 0(/) may be due to nonlinearities rather than to varia- 
tion of some parameter. However, the argument K'x is suppressed in $(/).] 
As before, a tentative Lyapunov function V = x'Px is adopted, and V 

is evaluated to be 



—x'[KRK' + Q + FG(0 - \)K' + K{<b’ ~~ 1 )G'P]x. 

Asymptotic stability follows if V is negative definite, which follows if 

I - [A(d> - I)B' + B(0' - I) A'] >el> 0 
in terms of the earlier definitions for A and B. In turn, this inequality holds if 

O - / O' 

0 <D ' - / 



; r 

'’'max L 



E] < 1 - €. 



(13.3-10) 



Rewriting of this inequality in terms of specific bounds on O seems very 
difficult. However, the inequality may be of some benefit practically. 

Examples have shown for the scalar-input case that the bounds of (13.3-3) 
and (13.3-4) are highly conservative; less conservative bounds are derivable 
by use of a Nyquist plot in conjunction with the Circle Criterion. For the 
multiple-input case however, the above theory, though incomplete, appears to 
represent the only feasible approach. Presentation of the theory appears 
justified on these grounds alone. 



Problem 13.3-1. For the single-input case, verify that the result (13.3-6) 
follows from (13.3-5). Show also that A min [E] and A max [£’] can never have the same 
sign. 



Problem 13.2-2. Show that the nonzero eigenvalues of AB ' + BA' are the 
same as those of 

~A'B A'A~ 

_B'B B'A_ 

Problem 13.3-3. Using the notation of the section, show that when 

K = -PG 

(the standard optimal regulator case) and Q > 0, the sector bounds [k i, k 2 ] satisfy 
K { < ■£, k 2 ~ oo. [Hint: First show that the inequality K'[Q -{■ KK'Y X K < / must 
be satisfied. Show also that K'( a/ -1- KK')~ l K = K'K{aI + K r K)~\ and use the 
fact that ( Q KK') > (a/ + KK ) for some a.] 

Problem 13.3-4. Suppose x = Fx ~\- gu is a prescribed single-input, time- 
invariant system, and u — k'x is a control chosen to stabilize the system; note 
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that u = k'x is not assumed to result from an optimal design procedure. Show how 
sector bounds for nonlinearity tolerance can be calculated in terms of an arbitrary, 
positive definite £), and a matrix P satisfying 

P(F + gk r ) + (F + gk'YP + kk' + Q - 0. 

The sector bounds depend on the choice of Q. Can you suggest how Q might be 
chosen ? 



13.4 DYNAMIC CONTROLLERS 



In this section we consider, somewhat briefly, the optimal design of 
dynamic controllers for linear regulators where constraints are imposed on 
the order of the controller. As may be inferred from the theory of earlier 
chapters, two possible approaches may be used. In the first approach, some 
of the system states are estimated with a dynamic estimator; these state esti- 
mates are then used in a nondynamic state feedback control law. The state 
estimator and nondynamic control law together constitute the optimal 
dynamic controller for the original system, and both must be optimally 
designed. In the second approach, a performance index is taken, which 
includes derivatives of the control variable; the plant is turned into an 
augmented system by the insertion of integrators at the input, and the 
optimal feedback law for the augmented system is derived. Constraints 
requiring output feedback will carry over from the original system to the 
augmented system, but the augmented system feedback law can also include 
feedback of the original system input (which becomes an augmented system 
state). The theory of earlier sections of this chapter, of course, applies to the 
selection of the augmented system feedback law, which can then be translated 
into a dynamic law for the original system. 

Design using estimators: the estimation problem. We shall consider the 
case of an //-dimensional, time-invariant, single-input, single-output plant 

x = Fx + gu y — h'x . (13.4-1) 

We also consider a state estimator having state equations 

* = /> + g u u + g le y (13.4-2) 

where the dimension of F e is less than or equal to the dimension of F. If 
the dimension of F e is the same as that of F, estimators that are optimal in 
the presence of noise may be designed; if F e has dimension one less than that 
of F, certainly a full estimate of x is achievable. Here, we are more interested 
in the case when the dimension of F e is less than that of F by two or more 
(perhaps because of a design constraint on the complexity of the controller). 
In any case, the vector w is required to approach asymptotically a vector 
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of the form H[x , for a certain H x . That is, we require the response of the 
system (13.4-1) and estimator (13.4-2) to approximate the response of the 
system 

x = Fx + gu y a — \h H x ]'x (13.4-3) 

for some matrix H x . 

For w to be an estimate of H\x y it is clear, from rewriting (13.4-1) and 
(13.4-2) as 

(w - H\x) = F e (w — H\x) +(g ie — H\g)u 

+ (g 2e h f - H\F + F e H\)x, 

that F e must be chosen such that the system z — F e z is asymptotically stable, 
and g lei g 2e , F c , and H l must satisfy 

(gu - g) - 0 (g 2 e h f - H\F + F e H\) - 0. (13.4-4) 

If F, /;, F f , and g 2f , are specified, the second of Eqs. (13.4-4) becomes a 
linear matrix equation for the unknown H { . It is always solvable, as described 
in Appendix A, if Fand F e have no eigenvalues in common. Once is deter- 
mined, g le follows immediately from the first of Eqs. (13.4-4). 

The question remains as to how F e and g 2e should be chosen. Some 
immediate restrictions on these matrices are that F e should have eigenvalues 
sufficiently far in the left half-plane to ensure fast estimation, but not so far 
in the left half-plane that noise becomes a problem; furthermore, a choice of 
F c and g 2c should be rejected if it leads to scalar outputs present in H\x not 
being linearly independent of the output h' x . (Otherwise, the estimator is 
needlessly complex — unless, of course, there is some advantage associated 
with filtering of contaminating noise.) But in the final analysis, the selection 
of F c and g 2c is determined by the resulting //,, which, in turn, is linked to the 
control problem that we consider now. 

Design using estimators: the control problem. Let us suppose that the 
estimator dimension is fixed and, for the moment, that the estimator itself 
is fixed. We consider temporarily the system ( 1 3.4-3) ; with the aid of the theory 
of previous sections, an optimal control law of the following variety can be 
formed: 

U*^[k\ k' 2 ]y a . 

Now, the control law u — k\y + k' 2 w is certainly a good candidate to 
consider for controlling the original system (13.4-1) augmented with the 
estimator (13.4-2). The dynamic controller for this case has the state equa- 
tions 

W — (F„ + g u k' 2 )w + (g 2e + g u k\)y 
u — k\y + k' 2 w. 



(13.4-5) 

(13.4-6) 
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We expect that this controller can achieve a response that becomes closer to 
that of a controller consisting of a standard Luenberger observer together 
with standard state-variable feedback as the rank of [h H x \ approaches n. 
For the particular case when rank [h N\] is n 9 the preceding dynamic con- 
troller has the form of the Luenberger estimator, which estimates H\x, fol- 
lowed by a constant control law, which is readily derived from the standard 
optimal state feedback law as follows. If w* — k'x is the standard optimal 
law, then the gains [k\ k' 2 ] are given from 




k’ 2 ] - k 1 



li 



The derivation of the estimator parameters such that rank [h H x ] — n is, 
of course, the difficult part of standard estimator design, as we have seen in 
Chapter 8 where the problem is approached from an alternative viewpoint. 

We now suppose that rank [h H { ] is not necessarily n , and that the 
estimator is no longer fixed. Our theory so far has avoided the important 
question as to what are the best values that can be chosen for g 2e and F e . 
Since different values of these matrices lead to different values of H u and, 
in turn, to different optimal control laws and performance indices, the real 
problem becomes one of choosing g 2e and F e so that the performance index 
is minimized [Note: The performance index measures the performance of 
(13.4-3), not (13.4-1), or (13.4-1) and (13.4-2) together. We obtain a control 
law that exactly minimizes this index, and then construct from it a dynamic 
control law for (13.4-1), which is not actually optimal for (13.4-1) but obvi- 
ously has the right sort of properties.] It turns out that we can extend the 
algorithm used for calculating the optimum k\ and k l9 knowing H l9 to incor- 
porate the determination of k , , k 2 , g le9 and F e within the required constraints . 
Problem 13.4-1 asks that the case of a single-input, single-output plant with 
first-order compensator (i.e., w is a scalar) be studied in detail. 

Since the writing of this section, an approach has been developed to the 
design of controllers which uses a performance index involving both the con- 
troller parameters and states [7]. The advantage of the approach to the 
design of dynamic controllers just given when compared with the approach 
of [7], and for that matter the approach next described, is that the quadratic 
form ( u'Ru + x'Qx) occurring in the performance index remains unchanged 
as different controllers, including controllers of different dimension, are 
considered. 



Design using control derivatives in a performance index. In the latter 
part of this section on dynamic controllers, we consider the approach that 
combines some of the ideas of Chapter 10 with the results of earlier sections 
of this chapter. This particular approach consists in optimally controlling 
the original system when augmented with dynamics at the inputs, using a 
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nondynamic control law. The optimal augmented system with a nondynamic 
controller is then interpreted as the original system with an optimal dynamic 
controller. The index minimized for the augmented system, when interpreted 
in terms of an index for the original system, includes cost terms involving 
derivatives of the original system inputs. 

One minor difficulty with this approach is that as the order of the com- 
pensator or controller changes, the form of the performance index associated 
with the original system must change, and thus there is no way of comparing 
two controllers of different order. This is not the case for the estimator 
approach discussed in the earlier part of this section. 

We now illustrate the linkage of the ideas of Chapter 10 and the results 
of earlier sections of this chapter with an example taken from [8], (Inciden- 
tally, the problem considered in the example cannot be solved using the esti- 
mator approach. The system considered has a fixed dynamic structure that 
cannot be viewed as an estimator in series with a nondynamic control law.) 



Example. System design using a fixed structure controller. Suppose we are given 
a single-input, single-output plant 

x — Fx -f gu y = h'x. (13.4-7) 

Let us suppose also that we wish to control the system using a fixed structure 
controller having equations 



u = 





y dt -f k 2 y + k 2 y 



(13.4-8) 



where the constants k u k 2 , and k 3 are adjustable parameters. This controller 
structure is often referred to as a three-term controller for obvious reasons, and 
has found wide application for stabilizing industrial control systems ; in practice, 
the derivative in (13.4-8) is obtained approximately. 

We desire to select the parameters k = [k u k 2 , k 3 ] so that the closed-loop 
system stability properties are as close as possible to those of a standard regu- 
lator. As a first step in formulating the problem in optimal control terms, we 
express the control law (13.4-8) as a state feedback law using (13.4-7). We 
have from (13.4-7) that 

y h'x y = h'Fx + h'gu y = h'F 2 x + h'Fgu h'gu 
which means that (13.4-8), when differentiated, may be written 



(1 - k 3 h'g)u - 0 kzh'F 2 + k z h'F + k,h')x - (k 3 h'Fg 4 k 2 h'g)u 
Using the notation k as a normalization of k as follows. 





A 

~kf 




kf 


A 

k — 


A. 

A: 2 


= (l -k 3 h'g)-' 


k i 




A 




J.'3_ 



we may write the preceding control law as 

u = k'[h F'h ( F') 1 h]'x + k'[ 0 g'h g'F'h]'u . 



0 . 



(13.4-9) 



(13.4-10) 
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We now recall, using the ideas of Chapter 10, that this is nothing other 
than a state feedback law for the system (13.4-7), augmented with an integrator 
at its input. That is, (13.4-10) may be written as 

u a = k' a x a (13.4-11) 

where 







X 

_ U _ 




h 

_0 



F'h 0 F'Yh 

g'h g'F'h 




(13.4-12) 



The augmented system equations are given from (13.4-7) and (13.4-12) as 



where 



x a = F a x a -f g a ii a 






(13.4-13) 



So far, all we have done is show how the three-term- controller can be 
viewed as a state-variable feedback law for the original system augmented with 
an integrator at its input. If the gain elements k u * 2 , *3 of a three-term con- 
troller are specified, then k may be calculated by using (13.4-9); also, the state 
feedback gain k a for the augmented system (13.4-13) may be calculated by using 
(13.4-12). 

Now we are in a position to apply the results of this chapter to the aug- 
mented system. A performance index for the augmented system may be defined 
and then k u * 2 , and k 3 adjusted until this index is minimized. 

We now give further details for the case when the index is 

V = tr [P] 

where 



P - r exp l(F a + g a k'Yt](Q + kjc' a ) exp [(F a + g a k')t] dt 

J 0 

for some positive definite symmetric Q. For this case, certainly dVjdk a can be 
calculated using results of earlier sections. In fact, it is not hard to calculate 
dVjdk rather than dVfdk^ using minor extensions of the earlier results. 
Setting this to zero gives necessary conditions for optimality. (See Problem 
13.4-2.) 

For the case when 
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the optimal feedback law has been calculated to be 

*,=0.2 /c 2 =2.21 *3 = 1.38. 

The minimum index is V* = 7.64. This may be compared to that for the stand- 
ard optimum (augmented) regulator, where V* =6.31. Using the results of 
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the previous section, we calculate the sector in which nonlinearities may be 
tolerated in the plant input transducers as [k u k 2 \ where 

K\ — 0.7 k 2 = 1.3. 

(Note that the theory of Chapter 10 has been assumed, which notes that non- 
linearities that can be accommodated at the augmented plant input can also 
be accommodated at the original plant input.) These bounds are actually very 
conservative, as may be checked. 

We comment that it is somewhat surprising that so much theory is neces- 
sary to select “optimally” the parameters of a three-term controller — particu- 
larly since in practice three-term controller parameters are selected for quite 
complex systems with often very little effort and quite good results [9]. 

In conclusion, we emphasize that many of the results discussed through- 
out the chapter are recent, and, as we have seen, there is still considerable 
scope for further developments. In fact since the writing of this section, a 
simpler result has been developed (in collaboration with D. Williamson and 
T. Fortmann) which is of considerable assistance in determining minimal 
dimension controllers — see Problem 13.4-3. 



Problem 13.4-1. Consider the case of a single-input, single-output system 

x = Fx + gu y = fix 

with a first-order estimator at its output, having state equations 

W =/eW + g\ e U + gle y 



where 

gu=h\g g 2e h' - h\F + f e h\ =0 

and z — f e z is asymptotically stable. Associated with this system is the following 
system 

x = Fx + gu y a = [// hi(g 2ei f e )]'x 

where, according to the theory of this section, the estimator states w approach 
h\x asymptotically. Define a suitable performance index of the form V = tr [P] 
for this system, and evaluate dVjdf e . Indicate how this result can be used in deter- 
mining an optimal dynamic controller. 



Problem 13.4-2. For the three-term controller problem, derive the neces- 
sary conditions for optimality. That is, find an expression for dVjdk a and set this 
equal to zero. 



Problem 13.4-3. Consider the /zth order system (13.4-1) and p\h order con- 
troller (13.4-5 and 13.4-6) where, using the notation of Chapter 8, det (si — F e ) = 
s p + a^ -1 -f . . . + a l and T ] FT { ~ 1 is in transpose companion matrix form for 
T x given in (8.3-10). With [F e , k 2 ] completely observable and i = F e z asymptotically 
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stable show that a necessary and sufficient condition for u to be an estimate of k'x 



for some fixed k is that S{T x ~ x )'k = 


0 with 
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[Hint: First show that a sufficient condition is that k = hk 1 -f //jA 2 for some H\ 
which satisfies (l 3.4-4). Next show that S[H , /i] = 0, by using the Cayley-Hamilton 
theorem and the fact that (13.4-4) constrains H x to have the form = [h x F e h x 
. . . F e n ~ l hi]. Assuming that h { is chosen so that [F e , h x ] is completely controllable, 
conclude the result. The multiple output case results in similar conditions]. 
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TWO FURTHER TOPICS 



14.1 DISCRETE TIME SYSTEMS 

This section is devoted to a brief exposition of linear regulator design for 
discrete time systems. Our starting point is the state-space equation 

x(t + 1) = F(t)x(t) + G(t)u{t) x{t Q ) given (14.1-1) 
and the performance index 

K(.r(/ 0 ), «(•), r 0 ) = 2 [x\t)Q(t)x(t) + u'(t — 1 )R(t)u(t — 1)]. 

t = t 0 + 1 

(14.1-2) 

In these equations, x(t) is the state at time t and u{t) the control at time t. 
Generally, but not always, t is assumed to take on integer values. The plant 
(14.1-1) is initially — i.e., at time t 0 — in state x(t 0 ), and the aim is to return the 
plant state to the origin, or a state close to the origin. To do this, we set up 
a performance index (14.1-2), in which Q(t) and R(t) are nonnegative definite 
symmetric matrices. [Note: We do not assume R(t) to be positive definite.] 
The performance index has the property that “large” values of the state will 
tend to make the performance index large. Hence, by choosing the control 
sequence u(t Q ) y u(t 0 + 1), . . . , which minimizes the performance index, 
we can expect to achieve the desired regulator effect. 

It might be thought curious that R(t) is not positive definite, since in the 
corresponding continuous time performance index, the corresponding 
matrix is positive definite. In the latter case, the presence of the matrix rules 
out the possibility of using an infinitely large control to take the state to zero 
in an infinitely short time. In the discrete time case, it is not possible to take 
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the state to zero in an infinitely short time, and the possibility of an infinitely 
large control occurring does not even arise. Hence, there is no need to prevent 
such a possibility by using a positive definite R(t). 

We shall first solve the optimization problem for the case of finite T. 
Then, with additional assumptions, we shall cover the infinite T case, with 
special reference to time-invariant plants. Finally, we shall remark on such 
matters as stability, tracking, etc. 

For finite T, we shall show that the optimal control is a state feedback 
law, and that the optimal performance index is quadratic in the initial state 
x(/ 0 ) — results that are, of course, analogous with the corresponding con- 
tinuous time results. 

The route to a derivation of the optimal control is via the Principle of 
Optimality. Thus, if until time t optimal controls w(/ 0 ), u(t Q + 1), . . 
u(t — 1) have been applied, leading to a state x(t), then the remaining terms 
in the optimal control sequence, u(t ), u{t + 1), . . . , u(T — 1) must also be 
optimal in the sense of minimizing K[x(r), w(-), t]. 

Now let V*[x(t), /] denote the optimal performance index associated 
with an initial state x{t) at time t . Then, by the Principle of Optimality, 

V*[x(t ), /] = min {[F(r)x(r) + G(t)u{t)]' Q(t + l)[F(/)x(/) + G(t)u(t)] 

uit) 

u ( f)R(t -\~ l)w(r) 

+ V*(F(t)x(t) + G(t)u(t ), t + 1)} 

= min { u\t)[G'(t)Q(t + \)G(t) + R(t + 1 )]«(/) 

uit) 

+ 2 x'(t)F'(t)Q(t + l)G(t)u(t) + x’{t)F'(t)Q(t + 1 )F(t)x(t) 

+ V*(F(t)x(t) + G{t)u{t ), / + 1)]. (14.1-3) 

Bearing in mind the corresponding continuous time results, it would 
be reasonable to guess that F*[x(r), /] would be of the form x\t)P{t)x(t). Since 
it proves convenient to make use of this result almost immediately, we build 
into the following argument an inductive proof of the result. 

We require first the following assumption. 

Assumption 14.1-1. For all f, G'(t - \)Q(t)G(t ~ 1) + R(t) is positive definite. 

This assumption may, in fact, be relaxed. However, when it does not 
hqld, the optimal control law becomes nonunique (although still linear), 
and we wish to avoid this complication. Notice that the assumption will 
very frequently hold, e.g., if R(t) is positive definite for all /, or if Q(t) is posi- 
tive definite for all t and the columns of G(t) are linearly independent, etc. 

With Assumption 14.1-1, it is easy to evaluate the “starting point” for 
the induction hypothesis — viz., V*[x(T — 1), T — 1]. We have 

V(x(T - 1), u(.f T ~ 1) = x'(T)Q(T)x(T) + u\T ~ 1 )R{T)u{T 1) 
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and, in view of the system equation (14.1-1), this becomes 

V(x(T - 1), w(.), T - 1) - x(T~ 1 )F\T- 1 )Q(T)F(T~~ 1 )x(T - 1) 

+ 2 x\t- i )F'(r- i)0(r)G(r- \) u ( t - i) 

+ u\t — i)0(r)C7(r- i) 

+ tf(7>(r- l). 

Evidently, the control z*(^ 0 that minimizes this performance index is 

a linear function of x(T — 1) — i.e., 

u*(T- 1) = K\T~ 1 )x(T- 1) (14.1-4) 

for a certain matrix K(T — 1). Moreover, the resulting optimal index 
V*(x(T — 1), T — 1) becomes quadratic in x(T — 1) — i.e., 

K*(x(r - 1), T - 1) = x\T - \)P(T - 1 )x(T - 1) (14.1-5) 

for a certain nonnegative definite symmetric P(T — 1). The actual expressions 
for K(T - 1) and P(T ~ 1) are 

K\T- 1) - —[G'(T — 1 )Q(T)G(T- 1) 

+ R(T)]'G'(T - 1 )Q(T)F(T- 1) (14.1-6) 

P(T - 1 ) = F'(T 1 ){Q(T)~ Q(T)G(T — 1 )[G'(T - X)Q(T)G(T - 1) 

+ R(T)Y'G\T- 1 )Q(T)}F(T- 1). (14.1-7) 

We now turn to the calculation of the matrices K{t ), determining the opti- 
mal control law', and P(t), determining the optimal performance index, for 
arbitrary values of t. As part of the inductive hypothesis, we assume that 
K*[x(f + 1), t + 1] — x(t + 1 )P(t + l)x(t + 1) for a certain matrix P(t + 1). 
By proving that K*[x(/), t] is of the form x\t)P(t)x(t) for a certain P(t), we 
will have established the quadratic nature of the performance index. [Of 
course, the expression for V*(x{T — 1), 7’ — 1) derived in Eq. (14.1-5) serves 
as the first step in the induction.] 

Applying the inductive hypothesis to (14.1-3), we have 

K*[x(/), t] = min [u(t)[G'{t)Q(t + 1)6(0 + R(t + 1 )\u(t) 

u(t) 

+ 2 x\t)F\t)Q(t + \)G(t)u(t) + x\t)F\t)Q{t + \)F{t)x{t) 

+ x\t)F\t)P{t + 1 )F(t)x(t) + 2x\t)F\t)P{t + 1 )G{t)u{t) 

+ u\t)G\t)P{t + \)G(t)u(t)}. 

Again, the minimizing u(t), which is the optimal control at time /, is a linear 
function of x(t), 

u*(t) = K'(t)x(t) (14.1-8) 

and the optimal performance index K*(x(0> 0, resulting from use of «*(/), 




I 



5ec> 74.7 DISCRETE TIME SYSTEMS 331 

is quadratic in x(0 — i.e., 

F*(x(/), 0 = x'(t)P(t)x(t). (14.1-9) 

The expression for K\i) is 

K\t) - ~[G\i)Q{t + 1 )G{t) + R{t + 1) 

+ + 1 )G(t)V'[G(t)Q(t + 1 )F(t) + G'(t)P(t + 1 )F(t)] 

= + DG(0 + R(t + 1 )Y l G\t)Q{t + l)F(t) (14.1-10) 

where Q(t + 1) = Q(t + 1) + P(t + 1). (14.1-1 1) 

The expression for P(t) is 

m = F\t){Q(t + 1 )-Q(t + \)G(t)[G\t)Q{t + 1 )G(t) 

+ R{t)Y l GXt)Q(t + 1 )}F{t). (14.1-12) 

Observe that (14.1-1 1) and (14.1-12) together allow recursive determina- 
tion of P{t) — i.e., determination in sequence and starting with P(T — 1) 
as given in (14.1-7) of P(T — 2), P(T — 3), ... . Equation (14.1-10) expresses 
the optimal feedback law in terms of known quantities, and the members of 
the sequence P(T ~ 1), P(T — 2), .... 

Thus, to solve a discrete time optimization problem of the sort posed, 
we must compute the matrices P(t) by starting at the endpoint and working 
backward in time; from these, the optimal gains must be found. 

We shall now consider the infinite-time problem, obtained by letting the 
final time Tin the performance index (14. 1-2) go to infinity. The performance 
index then becomes 

V(x(t 0 ), «(•), t 0 ) = lim 2 [x\t)Q{t)x(t) 

T—HX* t — t Q + 1 

+ u’(t - l)R(t)u(t - I)]. (14.1-13) 

To guarantee that the optimal performance index is finite, we shall require 
this assumption. 

Assumption 1 4. 1 -2. For each /, the pair [F(t), G{t)] is completely controllable — 
i.e., for any state *(/) there exists a time t x > t and a control defined on [/, t x ] 
with the property that application of the control leads to x(t x ) — 0. 

This assumption is not necessary to guarantee finiteness of the optimal per- 
formance index, but it may be shown to be sufficient (see Problem 14.1-1). 

In the same way as was done for continuous time systems, we define 
x'(t)P(t, T)x(t) to be the optimal performance index associated with (14.1-1) 
and (14.1-2); it is then possible to show that P(t, T) is monotonically increas- 
ing with T, and bounded above for all T. We conclude the existence of 

lim P(t , T) - P{t). 



(14.1-14) 
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Moreover, it is easily shown that 

F*[a(/), t ] = x'(t)P(t)x(t) (14.1-15) 

when (14. 1-1 3) is the performance index. Furthermore, P(-) satisfies the recur- 
sion relation (14.1-12) — i.e., knowing P(t : ) for any t i9 we can use (14.1-12) 
to compute successively P(t x 1), P(t { — 2), ... , and the optimal control 
law is given by (14.1-10). 

For the case of constant F, (7, Q y and R , P(t) becomes independent of 
t. We may then compute P via the formula 

lim P(t, T) — P (14.1-16) 

/ — * — 

using relation (14.1-12) recursively. [Equation (14.1-16) actually provides a 
more convenient approach to the evaluation of P than does Eq. (14.1-14).] 
The matrix P is also a solution of the “steady state” version of (14.1-12) — i.e., 

P ~ F’[{Q + P) ( Q + P)G[G\Q + P)G + R]~'G'(Q + P))F. 

(14.1-17) 

FTowever, this equation does not lend itself to the evaluation of Fin the same 
way as the corresponding continuous time equation. 

The control law in this case also becomes constant, and we see from 
(14.1-10) that it is given by 

K' = ~[G\Q + P)G + R] X G\Q + P)F (14.1-18) 

with the associated closed-loop system being 

x(t + 1 ) = (F + GK')x(t). (14.1-19) 

As for the continuous time case, an assumption guaranteeing asymptotic 
stability of the closed-loop system is as follows: 

Assumption 14.1-3. With D any matrix such that DD' — Q, the pair [F, D] 
is completely observable, or, equivalently, the equation D'F k x 0 — 0 for all k 
implies x 0 --= 0. 

Many of the remarks and extensions applicable to continuous time 
problems carry over to the discrete time case. For example, one may pose 
a tracking problem [aimed at finding a control to make the system state x(-) 
track a desired trajectory *(•)]• This is done by using a performance index of 

the form 

K(*0 0 ), «(•), t u ) = 2 {WO - x{t)\ Q{t)[x{t) - .v(0] 

t = t 0 -\ 1 

+ u\t - 1 )R(t)u(t 1)}. (14.1-20) 

The optimal control turns out, as in the continuous time case, to consist 
of a linear feedback term and an externally applied term. (See Problem 

14 . 1 - 5 .) 
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Two primary references to the quadratic minimization problem for 
discrete time systems are [1] and [2]. More recently, lengthy discussions on 
this and related matters have appeared in [3] and [4]. For convenience, we 
shall summarize the principal results. 

Finite time regulator . Consider the system 

x(t + 1) = F(t)x(t) + G(t)u(t) x(t Q ) given. (14.1-1) 

Let Q{t) and R(t) be nonnegative definite matrices for all /, with 
G\t ~ \)Q(t)G{t — 1) + R(t) nonsingular for all t. Define the perfor- 
mance index 

V(x(t 0 ), u(-) 9 1 0 ) - £ [x\t)Q(t)x{t] + u'(t - 1 )R(t)u(t - 1)]. 

/-r 0 + I 

(14.1-2) 

Then the minimum value of the performance index is 

V*(x(t 0 ), /„) = x’(t 0 )P(t 0 , T)x(t 0 ) 
where P(t, T) is defined recursively via 
P(t, T) = F'(t)\Q(t + 1) - Q(t + 1 )G(t)[G'(t)Q(t + 1 )G{t) 

+ RU + 1 )Y'G'(t)Q(t + l)}F(t) P(T, T) - 0 (14.1-12) 

and 

do + !)= QO + l) +P0 + lj). (14.1-11) 

The associated optimal control law is given by 

u*(t) = ~[G'0)QO + l)G(f) 

+ RO + 1 )Y'G'0)Q0 + 1 )F(t). (14.1-10) 

Infinite-time regulator. Assuming now that 7 — > oo, and that the pair 
[/■'(/), G(r)] is completely controllable for all t, then 

P(t) = lim P{t, T) (14.1-14) 

exists, and the optimum value of the performance index (14.1-2), with 
T replaced by infinity, is x'(t 0 )P(t 0 )x{t 0 ). The matrix P satisfies the recur- 
sion relations (14.1-12) with 

Q{t + 1) - Q(t + 1) + P(t + 1) (14.1-21) 

and the optimal control law is given by (14.1-10). 

Time-invariant regulator . When F , G, Q , and R are constant, P is con- 
stant and may be obtained via 

P - lim P(t, T ). 



(14.1-16) 
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The optimal control law is also constant, being given by 

“* = ~{G\Q + P)G + R]-'G'(Q + P)F. ( 14 . 1 - 18 ) 

With D any matrix such that DD' = g, complete observability of the 

pair [i% D] is sufficient to guarantee asymptotic stability of the closed- 

loop system. 

Problem 14.1-1. For the case of constant F , G, Q , and R with Q positive 
definite, show that a necessary and sufficient condition for the optimal perform- 
ance index associated with (14.1-13) to be finite, assuming the optimal index exists, 
is that no uncontrollable state fails to be asymptotically stable. 

Problem 14.1-2. Find the optimal control law and optimal performance 
indices for the system 

x(t + 1) = x(t) 4- u(t) x(0) given 

[where *(•) and w(*) are scalar quantities] with the two performance indices 

£ [2 x 2 (t) + u\t - 1)] 

t= i 

and 

S [ 2 x 2 (0 4 ~ u 2 (t — 1)]. 

/= i 

Problem 14.1-3. When Assumption 14.1-1 fails — i.e., G'(t — 1 )Q{t)G{t — 1) 
b R{t) is singular for some t — in the case of single-input systems this quantity is 
zero. Now the inverse of this quantity occurs in the formulas for K\T — 1) and 
P(T — 1) in (14.1-6) and (14.1-7). Show, by examining the derivation of the optimal 
control u\T - 1), that if G\T - 1 )Q(T)G(T - 1) 4 R(T) is zero, u*(T - 1) = 0 
will be a value, but not the unique value, of the optimal control, and that the inverse 
may be replaced by zero. Show that for t < T ~ 1, if G\t — 1 )Q(t)G(t — 1) 4 
GV ~ 1 )P(t)G(t - 1) 4" R(t) is zero, the inverse of this quantity in the expressions 
for K(t ) and P(t) may be replaced by zero. 

Problem 14.1-4. Given an /?-dimensional system, show that the optimal 
control minimizing the performance index 

V(x(t Q \ «(.), t 0 ) = xV 0 4 - n)Qx(t 0 + «), 
where Q is positive definite, will result in a deadbeat (x(t Q 4- n) = 0) response. 

Problem 14.1-5. Given the system 

*0 4-1) = F(f)x(t) 4- G(t)u(t) x(t Q ) given 
and the performance index 

n*Oo),«(-),G)= £ (wo - m\Q(t)[x{t) - *o)i 

-f u\t - 1 )R{t)u{t - 1)] 
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where x( ) is a sequence of states (the desired system trajectory), find the optimal 
control. 



14.2 THE INFINITE-TIME REGULATOR PROBLEM FOR 
TIME-VARYING SYSTEMS 



The purpose of this section is to present some advanced results concern- 
ing the infinite-time regulator problem associated with continuous time 
systems of the form 

x - F{t)x + G(t)u. (14.2-1) 

In earlier chapters, we generally restricted discussion of infinite-time problems 
to time-invariant systems, and merely alluded to the existence of correspond- 
ing results for time-varying systems. The results presented here serve to fill 
in some more details for time-varying systems, but do not appear to have 
the same practical significance as the corresponding time-invariant results. 
The primary reference is [5], 

The performance index associated with (14.2-1) we take to be 
V(x(t 0 ), w(-), t 0 ) = lim f [u'(t)R(t)u(t) + x'(t)Q(t)x(t)} cit (14.2-2) 

T’-.OO J 

where, as usual, R(t) is positive definite symmetric and Q(t) is nonnegative 
definite symmetric. We introduce the following assumption: 

Assumption 14.2-1. The pair [ F(t ), G(t )] is completely controllable for each t. 

As we know, it is then possible to write down the optimal control law and 
optimal performance index in the following fashion. We define P(t, T) to be 
the solution of 

— P = PF + F'P — PGR~ l G'P + Q P(T , T) = 0 (14.2-3) 

and we set 

P(t) = lim P{t , T). (14.2-4) 

J 1 — >oo 

(The existence of the limit may be established.) The optimal control law is 

u*(t) = K\t)x(t) = -R- l (t)G\t)P(t)x(t), (14.2-5) 

which yields the closed-loop system 

* = (F + GK’)x = (F — GR~ 1 G'P)x. (14.2-6) 

The optimal performance index is 

f 0 ] = x’(t a )P(t 0 )x(t 0 ). 

Questions such as the following then arise: 



(14.2-7) 
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1. When do K(t) and P(t) have bounded entries for all /? 

2. When is the closed-loop system of Eq. (14.2-6) asymptotically stable? 

We shall now list sufficient conditions (without proof, but with side remarks) 
for the boundedness of K(t), P(t ), and for a special form of asymptotic stabil- 
ity of the closed-loop system. This special form of asymptotic stability is 
exponential asymptotic stability: x — F(t)x is termed exponentially asymp- 
totically stable if for all /, / 0 , and x(f 0 ), there exist positive constants and 
a 2 such that 

< a, exp [ — 2a 2 (/ - f 0 )M*o)*(*o)- (14.2-8) 

As is evident from this definition, the size of x(t) is bounded above by an 
exponentially decaying quantity, the initial value of which depends on x(t 0 ). 

There are a number of conditions that, when taken together, will guar- 
antee the results we want. These occur here as Assumptions 14.2-2, 14.2-3, 
and 14.2-4, the set of conditions having a natural subdivision. We start with 
the simplest conditions. 

Assumption 14.2-2. The following matrices have entries that are bounded for 
all t : F(t\ G(t\ Q(t\ R(t), and 

In common with Assumptions 14.2-3 and 14.2-4, Assumption 14.2-2 
may not be necessary to guarantee the desired results in some situations. 
However, it is perhaps interesting to see how the conditions inherent in 
it may be relevant. For example, suppose F(t) and <7(0 are constant, and Q(t) 
= Qc ltL \ R(t) — Re~ 2ctt , where Q is constant and nonnegative definite, 
R is constant and positive definite, and a is a positive constant. We have an 
optimization problem which is readily convertible to a time-invariant prob- 
lem. Indeed, the minimization of (14.2-2), given (14.2-1), is equivalent under 
the identifications 

x — e~ at x 
u = 

to the minimization of 

f* Oo ^ 

( u Ru -f~ x Q: c) dt 

J to 

subject to 

x ~ (F — a I)x + Gu. 

With [/% G] completely controllable and Q positive definite, an optimal con- 
trol law u — K'x exists with K constant, and with F — a/ + GK f possessing 
eigenvalues all in Re[s] < 0. For the original problem, the optimal control 
law is u ■= K'x , but now the closed-loop system, x = (F + GK')x , may not 
be asymptotically stable! 

Thus, in this example are satisfied all the right sort of conditions that we 
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would expect would guarantee asymptotic stability of the closed-loop system, 
except for the boundedness of the entries of R~ x {t). Yet asymptotic stability 
may well not be present. 

The second condition introduces a new term, defined momentarily. 

Assumption 14.2-3. The pair [F(/), G(/)] is uniformly completely controllable [5]. 

Uniform complete controllability is an extension of the concept of complete 
controllability. Before we give a quantitative statement of the definition, the 
following qualitative remarks may help. We recall that a system is completely 
controllable at time / if, given an arbitrary state x(t) at time /, there exists a 
control and a time ^ > t such that application of the control to the system, 
assumed in state x(/) at time t, will lead to x(^) — 0. Uniform complete 
controllability requires, first, that there be complete controllability at 
every time /, and second, that no matter what t is, the time t A — t taken to 
get to the zero state should be bounded above. Third, the control energy 
required to transfer the state from x(/) to x(/j) — 0, as measured by 

J' u(r)u(r) dr , 

should be bounded above and below by positive constants independent of 
t , and depending only on x'(0*(0» or the size of x(t). That is, for all t and x(/), 
there should exist a control u(-) achieving the required state transfer, and 
such that 

a 3 x'(t)x(t) ^ ^ u (r)^(r) dr ^ 

for some positive constants a 3 and a 4 . 

However, this is not all. A system is completely reachable at time t if, 
given an arbitrary state a*(Y), there exists f 2 < t and a control w(-) such that 
with x(t 2 ) = 0, the control u(-) will take the system state to x(t) at time t. 
Three more qualifications of uniformly completely controllable systems are 
that they are completely reachable at every time /; that no matter what 
t is, the time / — t 2 is bounded above; and that the control energy effecting 
the transfer from xc(r 2 ) = 0 to x(/) should be bounded above and below by 
positive constants independent of /, and depending only on x'(t)x(t). 

A moment’s reflection will show that with F and G constant, complete 
controllability of [/% G ] implies uniform complete controllability. {For exam- 
ples of completely controllable but not uniformly completely controllable 
pairs [F{t\ G(t)), see, e.g., [6].} 

The precise quantitative definition of uniform complete controllability 
is as follows. Let <!>(*, •) be the transition matrix associated with x ~ F(t)x , 
and define 

/ ,) =- r ®(r 0 , t)G{t)G'(tW{t 0 , t) dt . 

J to 



(14.2-9) 
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Then the pair [F(?), C(r)] is uniformly completely controllable if two of the 
following three conditions hold (if any two hold, the third automatically 
holds): 

0 < a 3 (ff)/ < W(t, t + a) < a»/ < oo, (14.2-10) 

0 < a 5 (ct)/ < <t>(; + a, t)W(t, t + a)(S>'(t + a, t) 

< « 6 (ff)/ < oo, (14.2-11) 

and 

max | (<J>(r, t )), 7 | < a 7 (| / — r|) for all t, r. (14.2-12) 

». j 

In these equations, a 3 («X • • * , a 7 (*) are continuous functions of their argu- 
ments. 

One property of uniformly completely controllable pairs that proves 
useful is the following (see [6]). 

Lemma. If F(t) and G(t) have bounded entries, and if the pair [F(/), G{t)] is 
uniformly completely controllable, then for any K f (t) such that G(t)K'(t ) has the 
same dimension as F(t) and such that the entries of K(t) are bounded, the pair 
[F(t) + G{t)K'{t ), (7(f)] is uniformly completely controllable. 

It follows from this result, for example, that the commonly occurring pair 





0 1 0 • • 0 " 




~0" 




0 0 1 • • 0 
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Fit) = 


* ♦ » * « • 
• * « • m m 


G(t)r= 
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0 0 0-1 




* 




-a lit) -<z 3 (0 ■ • ~a„{t)_ 




_ 1 _ 



(14.2-13) 

is uniformly completely controllable so long as the aft) are all bounded. 

The third assumption required for the desired stability results is the fol- 
lowing. 

Assumption 14 . 2 - 4 . With D{t) any matrix such that D(t)D'{t) — Q(t ), the 
pair [F(f), D(t)J is uniformly completely observable . 

As might be imagined, the term uniform complete observability can 
most readily be defined as the dual of uniform complete controllability: 
£(0] is uniformly completely observable if and only if [F'(f), £>(/)] is 
uniformly completely controllable. 

Assumptions 14.2-2 through 14.2-4 guarantee the following results for 
the infinite-time regulator problem [5]: 
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1. The optimal control law gain K(t) has bounded entries for all /. 

2. The matrix P(t) satisfies 

a 8 / < P(t) < oc 9 / (14.2-14) 

for all t and for some positive constants a 8 and a 9 . 

3. The optimal closed-loop system [see Eq. (14.2-6)] is exponentially 
asymptotically stable, and x f (t)P(t)x(t ) is a Lyapunov function estab- 
lishing this stability property. 

This completes our discussion of the optimal regulator. We now turn 
to a related topic, the infinite-time Kalman-Bucy filter. We recall that we 
consider a plant 

x(t) = F(t)x(t) + G{t)u(t) + v(t) (14.2-15) 

y (t) = H'O)x(t) + w(t) (14.2-16) 

where E[v(t) v'(x)\ = Q(t)S{t — t), £[w(/)h>'(t)] = R(t)d(t — t), with £>(-)and 
w(-) independent zero mean gaussian processes. Furthermore, E[x(t Q )x'(t Q )) 
— P 09 £[x(f 0 )] = m, and x(t Q ) is independent of v{t) and vv(/) for all t. Finally, 
x(? 0 ) is a gaussian random variable. 

The state vector x(/) may be estimated, via an estimate x e {t ), available 
at time t . The equation describing the calculation of x e (t) is 

^x/t) = [F(t) + K e U)H'(t)]x f (t) - K e {t)y{t) 

+ G(t)u(t) x e (t 0 ) = m (14.2-17) 

and the matrix K e (t) is found as follows. Let P(t) be the solution of 

p = PF' + FP - PHR l H f P + Q P(t 0 ) - P 0 . (14.2-18) 

Then 



K e {t) = - P(t)H(t)R~'(t ). (14.2-19) 

Equation (14.2-17) describes the optimal filter, concerning which we 
ask the following question : Under what conditions is the optimal filter asymp- 
totically stable? In effect, we can present only sufficient conditions for expo- 
nential asymptotic stability. Although it is possible to derive them by studying 
an equivalent regulator problem, as we did in Chapter 8, we shall content 
ourselves here with merely stating the following result. Sufficient conditions 
for the optimal filter to be exponentially asymptotically stable are [7] as fol- 
lows: 



1. The entries of £( (), H(t ), Q(t ), R(t ), and R~ l (t) are bounded for 
/ > (or for all Mf t 0 — — co). 

2. The pair [£(0> //(f)] is uniformly completely observable. 

3. With D(t) any matrix such that D(t)D'(t) = Q{t ), the pair [F(t), 
D(f)] is uniformly completely controllable. 
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Problem 14.2-1. Prove that if [F(t), G(/)] is uniformly completely controllable, 
so is [F(0 + a/, G(/)] where a is a positive constant. Suppose that F(f), Gif), Q(t), 
and R{t) satisfy Assumptions (14.2-2) through (14.2-4) with, as usual, Q{t) nonnega- 
tive definite and R(t) positive definite. Show that the performance index 

f e 2at [u'(t)R(t)u(t) 4- x(t)Q{t)x{t)\dt 

J fo 

may be minimized by a control law of the form u(t) = K'(t)x(t) with the entries of 
K(t) bounded, and that the closed-loop system has degree of stability a. 
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PART V 



COMPUTATIONAL ASPECTS 




CHAPTER 




SOLUTION OF THE RICCATI 

EQUATION 



15.1 PRELIMINARY REMARKS 



The selection of an optimal control law and the design of optimal filters 
require the solving of a Riccati equation. The natural question arises as to 
how this solution procedure might be carried out. In this chapter, we present 
a number of different approaches to the solution computation, some of which 
are only applicable in restricted situations. 

It proves convenient to make several subdivisions of the problem of 
finding Riccati equation solutions, the basis for subdividing being the nature 
of the coefficient matrices of the Riccati equation and the time interval 
considered. Throughout this chapter, we restrict attention to the Riccati 
equation occurring in the control problem, and thus to a situation when a 
boundary condition will be given at t { with the solution desired for t < t x . 
The subdivisions are, then: 

1. Time-varying coefficients; t x < oo. 

2. Time-varying coefficients; t x — oo. 

3. Constant coefficients; t Y < oo. 

4. Constant coefficients; t x = oo. 

In all cases, it is possible to solve the Riccati differential equation directly 
by appropriately programming a digital (or, for that matter, analog) computer. 
This would appear to be a common procedure, despite the availability of 
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other procedures to be discussed in later sections. Solution via the digital 
computer this way naturally requires discretization of time. 

When t s — OO and direct solution of the Riccati differential equation is 
employed, cases 2 and 4 will be treated differently. For case 2, where the 
coefficients are time varying, we would use the fact that the solution P(t) 
of the Riccati equation for t x = oo is the limit of the solutions P(t, t { ) as t x 
approaches oo. Therefore, P(i) will be obtained approximately by choosing a 
large t l9 and the approximation will become better as t l is increased. In case 
4, where P(t) is a constant, the fact that the equation coefficients are constant 
guarantees that 

P(t) = lim P(t, t { ) = lim P(t , t x ). (15.1-1) 

lj — »oo f— * — oo 

Therefore, an arbitrary t x can be chosen, and the Riccati differential equation 
solution can be computed backward in time until a constant value is reached. 

Whenever computation of the solution is required over a large interval, 
the question must be considered as to how computational errors introduced 
at one point in the calculation will propagate. Reference [1] shows that if the 
closed-loop system resulting from application of an optimal control law is 
asymptotically stable, then the Riccati differential equation whose solution is 
used to define the optimal control law is computationally stable. That is, 
as an error propagates, it is attenuated, and there is no possibility of buildup 
of the error. We have earlier given conditions for asymptotic stability of the 
closed-loop system in the time-invariant case; more complex conditions apply 
in the time-varying case, as discussed in [1] and summarized in Chapter 14. 

Tied up with the question of the propagation of computational errors 
are the infinite-time problem questions of how large t x should be taken so that 
P(t, t x ) will be a good approximation to P(t), and how small t should be taken 
in the constant coefficient matrix case so that P{t , t x ) will be a good approxi- 
mation to P. A rough rule of thumb may be applied if the closed-loop system 
is asymptotically stable: In the constant coefficient case, t x — t should be 
several times the dominant time constant of the closed-loop system, and an 
appropriate modification of this statement holds in the time-varying coeffi- 
cient case. Since this time constant will not be known a priori, this rule of 
thumb is perhaps not particularly helpful, but it is perhaps helpful to know 
a priori that the sort of interval lengths t x ~ t involved are not long compared 
with other times that may naturally be of interest. 

Other ways of solving the Riccati equation will be discussed in the next 
two sections. In all four cases 1 through 4, it is possible to replace the problem 
of solving the nonlinear Riccati differential equation by the problem of 
solving a linear differential equation, and then computing a matrix inverse. 
This is discussed in the next section. The third section discusses a particular 
approach applying only to case 4. In this case, it is known that the Riccati 
equation solution is constant and satisfies an algebraic rather than a differ- 
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ential equation; direct solution procedures for the algebraic equation can 
then be used. 

The basic reference discussing the use of linear differential equations 
for solving Riccati equations is [2]. The theory behind this computational 
technique is given in [2], and earlier in, for example, [3] and [4], with simpli- 
fications in the constant coefficient matrix case appearing in [5] and [6], 
Techniques aimed at solving the steady state algebraic equation have been 
given in [7] through [11]. A recent survey comparing various techniques (and 
favoring a technique based on a result in [9]) may be found in [12]. 

In the fourth section of this chapter, we consider approximate solution 
techniques for high-order Riccati equations, which rely on the solving of a 
low-order Riccati equation and linear equations. 



15.2 RICCATI EQUATION SOLUTION WITH THE AID OF 
A LINEAR DIFFERENTIAL EQUATION 



For the moment, we restrict attention to the finite-interval problem. 
Thus, we suppose we are given the Riccati equation 

-^T = P(t)F(t) + F’(t)P(t) - P(t)G(t)R~ l (t)G'(t)P(t) + Q(t) 

P(t l ) = A. (15.2-1) 

We shall establish the following result (see, e.g., [2] through [4]). 



Constructive procedure for obtaining P(t). Consider the equations 
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~X~ 
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~x~ 
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dt 


Y 




Q -F’ _ 
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Jit,)] 







(15.2-2) 



where the explicit dependence of the matrices on t is suppressed. Then, 
provided that the solution of (15.2-1) exists on the interval [/, /,], the 
solution of (15.2-2) has the property that X''(t) exists and that 

P(t)= (15.2-3) 

Conversely, the solution of (15.2-1) exists on [t, t x ] if the solution of 
(15.2-2) has the property that X(o) is nonsingular for all a in t < <j < t i . 
The solution of (15.2-1) is given by (15.2-3). 



In the usual application of the Riccati equation to designing optimal 
regulators, the existence of P(t) for all t <C t x is assured. [Recall that sufficient 
conditions for this are as follows: Q{t) is nonnegative definite symmetric for 
all R(t) is positive definite symmetric for all /, and A is nonnegative definite 
symmetric.] Accordingly, we can normally forget about the existence remarks 
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in the preceding constructive procedure and concentrate on the actual con- 
struction. As is evident from the statement of the procedure, basically two 
operations are required. 

1. Solution of a linear differential equation, of matrix dimension twice 
that of the dimension of the matrix F{t). 

2. Matrix inversion and multiplication. 

We shall now verify the procedure. We do this by showing that the matrix 
Y(t)X~ 1 (t) satisfies the Riccati differential equation (15.2-1). It then follows, 
by the uniqueness property of solutions of differential equations with con- 
tinuous coefficients, that P(t) is given by (15.2-3). Formally, 

4-[ YX~'] = YX - YX 'XX -■ 
at 

- -QXX ~~ F f YX - 1 - YX l FXX l 
+ YX-'GR-'G'YX-' 

on using (15.2-2). Therefore, 

- J-[YX~ X ] = [YX~'\F + F’[YX-'] - [YX-'\GR-'G'[YX~'] + Q. 

This is the differential equation (15.2-1). Note, also, that the boundary 
condition in (15.2-2) yields immediately that y(r 1 )X'“ 1 (f l ) = A , which is the 
required boundary condition. Consequently, (15.2-3) is established. 

The preceding manipulations also show that if X^^a) exists for all 
a in [r, fj, a solution of (15.2-1) exists over the same interval. Thus, the second 
of the two existence claims in the constructive procedure statement is verified. 

Let us now check that existence of a solution P(t) to (15.2-1) guarantees 
that X'^t) exists. This will complete the proof of all claims made in the con- 
structive procedure statement. 

Let $(•, •) be the transition matrix associated with 

^ = [F(t) - G(t)R-'(‘)GV)P(t)]x. (15.2-4) 

Because P(t ) exists for / < <£(■, • ) is defined for all values of its arguments 

less than or equal to t x . We claim that 

X{t) = a>(t, c) Y(t) - P(tmu t x ) (15.2-5) 

satisfy (15,2-2) including its boundary condition. This is sufficient to prove that 
2f -1 (/) exists, since <& _1 (/, t { ) — ®(/ l5 t) is known to exist. 

To verify the claim (15.2-5), we have 

oi = [p(t) - G(t)R-'(t)G\t)P{t)m, t o 

= F(t)mt, ?,)] - Gw-'WGXtmmt, tji 
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Also, 

- -pu)Fum, /,) - no wcr, t t ) 

+ p(t)G{t)R-'( t )G'(t)P{tMt, r.) - g(o®a o 
+ P{t)F{tMt,t ,) - P(0G(0i?-‘(0G'(0P(0<l>(^, O 

= /,)] - F(/)[m OL 

Therefore, we have verified that, with the identification (15.2-5), the differ- 
ential equation part of (15.2-2) holds. Also, with the identification (15.2-5) 
and the boundary condition P(t x ) = A , we recover the boundary condition 
part of (15.2-2). The identifications (15.2-5) are therefore validated. 

An alternative expression for P(t) is available in terms of the transi- 
tion matrix associated with the equations (15.2-2). With F an n X n matrix, 
let the 2 n X 2 n matrix ©(*, •) be the transition matrix of (15.2-2) — i.e., 







-GR'G'l 




0(f, T) 



0(t, t) = / 2 „ for all r. 



Partition 0(f, t) into four n X « submatrices, as 



(15.2-6) 

(15.2-7) 



0 0 

0 = ^11 ^12 

_0 2 i ©22- 

Then (15.2-2) implies 

X(f) = ©life C) + ®i 2 (A t x )A 

Y(t) = ®2l('> C) + ® 22 ('> 

and so 

P(0 = [® 21 ('> C) + ©22<>> 'lKI[©ilO, t x ) 
+ ©i 2 (a 



(15.2-8) 



(15.2-9) 



For computing P(r) when F, <7, g, and P are constant, the advantage of using 
this procedure may be substantial, since an analytical formula is available for 
the transition matrix ©(*, •). Certainly this formula involves matrix expo- 
nentiation, which itself may be hard. But in many instances this may be 
straightforward, and in many other instances it may be tackled by one of the 
many numerical techniques now being developed for matrix exponentiation. 

To illustrate the foregoing procedure, we shall solve 
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with P(0) = 0, We first construct the equation for 0: 
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To carry out the matrix exponentiation required to represent 0 ana- 
lytically, we shall conduct a similarity transformation on the preceding 
coefficient matrix to obtain the associated Jordan form. The similarity trans- 
formation is constructed by first evaluating the eigenvalues of the coefficient 
matrix — which turn out to be +1 twice and —1 twice — and then finding the 
associated eigenvectors. There are actually only two proper eigenvectors, 
and the Jordan form turns out to have two nonzero elements on the super- 
diagonal. Once this is established, it is easy to compute the complete similarity 
transformation. We have 
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Writing this equation symbolically as 

AT - 77, 

then 

V te‘ 0 0 
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Now, T~ l may be computed to be 
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Therefore, 
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By Eq. (15.2-9), it then follows that 




' —e* -{- te‘ 
+e~‘ -I- te* 

— te ( f te 1 



te 1 — te* 1 



— e r — te* 
~he~* --- te~*_ 
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+(2 + V2)e"' — (v^2 + 



We can also evaluate lim P(t), which should be a steady state solution of the 

f — OO 

Riccati equation satisfied by P. To do this, we can delete terms involving 
e ‘ or te t in the preceding expression for P(t) as these become negligibly small, 
so long as the inverse in the formula for P(t) exists after this deletion. Thus^ 



lim P(t) ~ lim 
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(15.2-10) 

Indeed, the necessary inverse exists, and proceeding further with the calcula- 
tion, we have 



lim P(t) — lim 



4- te~* 

te '' 



e~* — te~ l _ 



1 

(3 + 2^T2)e~ 2t 



x (2 + a/2~ )e ' — (+fl 4- 1 )te r e * - te 1 

_e * (\t r 2^ + l)te * 2e~ t 4- te * 

(3 + 2VT)e' 2 ' 

__ ~2 — 3 — 2 

3 - 2^/2 6 - 4VT_ ’ 

It is readily verified that this constant matrix both satisfies the Riccati equa- 
tion and is a positive definite matrix. 

To provide an analytically solvable example, we restricted attention to 
a Riccati equation with constant coefficient matrices. Note, however, that the 
computation of the Riccati equation solution using a linear differential equa- 
tion certainly does not require constant coefficient matrices. 

References [5] and [6] offer certain comments applicable when the matrix 



(24 v 2 )e~ 2t e~ 2t ~ 
e~ lt 2e~ 2t 
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coefficients are constant. Reference [5] is concerned with setting up formulas 
for P(t) and the limiting P by the same diagonalization procedure as used 
previously, and with comparing the limiting / formulas with those obtainable 
by other techniques. Reference [6] extends these ideas and removes a com- 
putational problem which may arise. 

To understand the nature of this problem, consider the preceding exam- 
ple, and, in particular, Eq. (15.2-10). The 1-1 entry of the first matrix on the 
right side of this equation is e ~ ' + te~\ and as t — ► — oo, both summands 
become infinite. But they do so at different rates, and one might be tempted 
to neglect the e~ ( summand, or to overlook it in a numerical calculation. Yet 
to do so would prove fatal, as a major error would be introduced into the 
final answer. This difficulty is not peculiar to the present example either; in 
general, terms such as + oc 2 e~ Mt will be involved in an expression like 

(15.2-10), with t approaching minus oo. Although the term with the greatest 
value of will dominate, no other terms may normally be neglected. Acci- 
dental neglecting of the other terms— a strong possibility in numerical com- 
putation — will lead to a major error in the result. 

The difficulty is eliminated in the following way. To solve the equation 

-P = PF + F'P-PGR~'G'P + Q P(0) = A (15.2-11) 

with F, G, Q, and R constant and the other usual constraints, we start with 
the matrix 




- GR-'G’l 




(15.2-12) 



It is possible to show that this matrix has no purely imaginary eigenvalues, 
and that if X is an eigenvalue, so is —X (see [5] and Problem 15.2-4). Let 



~T U 



be a matrix with the property that 




T l MT 



'-A 0" 

- 0 A_ 



where A is a matrix that is the direct sum of 1 x 
or 2 x 2 blocks 



(15.2-13) 



(15.2-14) 
blocks [2,] with X, > 0, 



X, n t 

--Pi 

with X, > 0, Xt and n, always being real, and Jordan blocks of greater size 
again. Note that T is also real. Let the n x n matrices X{t) and Y{t) satisfy 




- GR'G'l 




~X~ 

Y 



(15.2.2) 
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or 




with 7(0) = /, 7(0) = A. Then define new n x n matrices X(t) and 7(0 by 



' 7 ' 

7 




(15.2-15) 



It follows that 







'-A 


0 




7 






0 


A, 




- 

„7j 



and, therefore. 



-f( 0 )- 




0 “ 


~x(ty 


y«)_ 




0 e A, _ 





(15.2-16) 



The boundary condition 7(0) = /, 7(0) = A implies by (15.2-15) that 
/ = T U X( 0) + r i2 (0)7(0) and A - T 2l X( 0) + T 22 Y( 0), or 

7(0) - RX{ 0) (15.2-17) 

where 



R = -t^2 - ^r 12 ]- i tr 21 - ^r u ]. (15.2-18) 

Now we shall find a new expression for P(t) — Y(t)X~ l (t), which we 
recall is the solution of (15.2-1). Using (15.2-16) and (15.2-17), we obtain 
immediately the following relation between 7(0 and 7(0* 

7(0 = e At Re At X{t). 

Now, using (15.2-15), we obtain 

PC 0 = (T 2l + T 22 e A( Re At )(T n + T l2 e At Re A T' (15.2-19) 

This is the desired new formula for P{t). A number of points should be 
noted. First, there are no growing exponentials as t approaches minus infinity 
appearing in the formula for P{t ) ; in fact, all exponentials decay. Second, when 
t approaches minus infinity, the limiting P(t) is given simply as 

P = lim P(t) - T 21 Tt}. (15.2-20) 

f -V — CO 

(It is possible to show that T u is always nonsingular.) Third, the limiting 
value of P(0 is independent of R and thus of the initial condition A of the 
Riccati equation. Fourth, from the point of view of simply computing P(t) 
rather than studying the theory behind the computation, the steps leading 
up to (15.2-19) are almost all dispensable. One starts with the matrix M of 
(15.2-12) and generates the matrix T of (15.2-13), which “almost” diagonalizes 
M [see (15.2-14)]. With the definition of R in Eq. (15.2-18), the formula 
(15.2-19) then gives P(t) and, inherently, lim P(t). Fifth, the formula (15.2-19) 

l — ► — oo 
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does not suffer from the computational disadvantage of the formula (15.2-9), 
upon which we earlier remarked. 

In the next section, we discuss more thoroughly the calculation of P . 
We shall also indicate an expression for P(t) in terms of P. 



Problem 15.2-1. Consider the Riccati equation 



— p — 2fp — f q 

with initial condition p(t 0 = 0. Solve this equation by the technique given in this 
section and compare your result with that obtained from 

dp 



: 



pit) 2 fp - (p 2 /r) + q 

Problem 15.2-2. Solve the Riccati equation 



= tx - t. 



p = p 


“0 0~ 


+ 


'0 r 


i 

p — -p 


- 1 ■ 




_1 0_ 


_o o_ 


r 


_o_ 



[1 0 }P 



P( 0) = 



‘0 O' 

.0 1J 

using the first method outlined in the text. 



Problem 15.2-3. Solve the following Riccati equation forward in time, 
obtaining the limiting solution: 



P = P 



0 0 ~ 
1 V"2~_ 






~0 -1 ~ 


P ~P 


— j — 


Lo \/~ t \ 




_0_ 



[1 



with boundary condition P( 0) = 0. 



0 ]P + 



"0 O' 
_0 1 _ 



Problem 15.2-4. Consider the matrix M of (15.2-12) with Q = Q' non- 
negative definite. Show that if X is an eigenvalue of M, so is —2. Can you show that 
there are no purely imaginary eigenvalues of Ml (Hint for first part: From an eigen- 
vector u satisfying Mu = Xu, construct a row vector v such that vM — - — Xv\) 



Problem 15.2-5. Using the method outlined after Eq. (15.2-11), solve the fol- 
lowing Riccati equation, which appeared earlier in the worked example: 



-P = P 




with P(0) = 0. 





0 ]P + 



~0 0 " 
-0 1 . 



15.3 STEADY STATE SOLUTION OF THE RICCATI EQUATION 

In this section, we further restrict the Riccati equation problem consid- 
ered. The coefficient matrices of the equation are constant, and we seek only 
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the steady state (and constant) solution P of the Riccati equation which is 
positive definite. (It is implicitly assumed throughout the following material 
that the controllability condition guaranteeing existence of P is satisfied, and 
that the observability condition guaranteeing positive definiteness rather 
than nonnegative definiteness of the limiting solution is also satisfied.) The 
actual equation satisfied by P is 

PF + F'P - PGR-'G'P + Q = 0 (15.3-1) 

which is an algebraic quadratic matrix equation. 

We shall present several different approaches to the solution of (15.3-1). 
One has already been more or less covered in the previous section, but we 
shall review it again here. This procedure is best discussed by Potter in [7], 
although it appears to have been essentially given in [8]. 

From the coefficient matrices in (15.3-1), we construct the matrix 




—GR'G'l 




(15.3-2) 



which can be shown to have the property that there are no pure imaginary 
eigenvalues, and the property that if A is an eigenvalue, so is —A. We then 
construct a matrix 




(15.3-3) 



that takes M almost to diagonal form, in accordance with the formula 



T~ l MT 



'-A 0" 
0 A 



where A is the direct sum of matrices [A,] with A,- > 0, matrices 



(15.3-4) 



"A,. ~ Ml ~ 

.Mi 

again with A , greater than zero, and Jordan blocks of greater size again. 
Then the desired matrix P is given by 

P=T 2l T^. (15.3-5) 

It is perhaps instructive to show directly that the quadratic equation (15.3-1) 
is satisfied by (15.3-5). (The direct proof that P is positive definite is a little 
more difficult ; since we are primarily interested here in presenting computa- 
tional procedures, we refer the interested reader to [7], In any case, the mate- 
rial of the last section provides an indirect proof.) To verify that (15.3-5) 
is a solution, we have from (15.3-4) that 



MT = 






and therefore 
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FT U ~ GR~'G'T 21 - -T n A 
~QT u -F'T 2l - -r 21 A. 

The first of these yields 

T 2l T;}F- T 2l TUGR-'G'T 2l T 7/ - -t^tut^at-,1 

and the second yields 

-q - f't 2{ t = -r^rr/TnArn 1 , 

whence 



“G - - T 21 TtIF- T u Ti}GR' l G'T u Til 

and the desired result follows. 

To illustrate the procedure, we consider the steady state equation 



D 0 ~ 




'0 1 ' 




" 1 " 




"0 0 " 


_1 -vT_ 


+ 


_o -yr_ 


P- P 


_ 0 _ 


[1 o ]P + 


0 1_ 



(The associated differential equation was solved as a worked example in 
the last section.) The matrix M becomes 




"0 

1 

0 

0 



0 -10" 
-vT 0 0 

0 0-1 
-1 0 vT 



The eigenvalues and eigenvectors of M were calculated in the last section. 
This calculation yields the equation 



with 



Thus, 



T~ l MT 



~-l 10 0" 

0-100 
0 Oil 
0 0 0 1_ 



1 1 1 -1 
VT + l -VT - 2 */2-\ vT - 2 
1 0-10 
1-111 





■ 1 

VT+i 



l 

-nTT-2_ 



2 - VT 3 - 2 */~ 2 ~ 

3 - 2VT 6 - 4VT 



which agrees with the result of the previous section. 
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A second procedure for solving (15.3-1), which is closely related to 
the preceding procedure, is given in [9]. Whereas the previous method requires 
computation of the eigenvalues of the matrix M and essentially its eigenvectors 
(which are more or less the columns of the matrix 7’), the procedure of [9] 
requires computation of the eigenvalues only. This may represent a saving 
in computation time, although there are aspects of the calculation which in 
places make it more unwieldly than the aforementioned procedure. 

We state the new procedure first, and then justify it. Starting with the 
matrix M of (15.3-2), we compute the eigenvalues of M and form that poly- 
nomial p(s ) whose zeros consist of the left half-plane (negative real part) 
eigenvalues of M. Suppose, for the sake of argument, that 

p(s) = s n + a n s n ~ l + * • * +a { (15.3-6) 

which has degree equal to the size of the square matrix F. We next construct 
the 2 n X 2 n matrix 



p(M) = M n + a n M n ~ 1 + • • 
Then P is uniquely defined by 



p(M) 





+ a J- 



(15.3-7) 

(15.3-8) 



To justify the procedure, let T be the matrix of (15.3-3), which achieves 
the pseudodiagonalization of (15.3-4). Then 



T~ x p(M)T — p(T~'MT) 



X- A) 0 - 

_ 0 p(. A)_ 

"0 0 “ 
_0 Pi al 



The last equality follows because the zeros of p(s) are the negative real 
part eigenvalues of M, which are also the eigenvalues of —A. The Cayley- 
Hamilton theorem then guarantees that p(— A) will equal zero. Now 



whence 



or 



p(M) 



~T ,, 


2V 




~T U 


t I2 - 




”0 


0~ 


T 2t 


^22_ 




T 2t 


^22- 




0 


P( A)_ 



p(M) 



1 u 

72 1 



o 



p(M) 



0. 



LTj.rr/j 

The method first discussed in this section shows that T 2l T;'; is precisely P, 
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the desired matrix, and hence (15.3-8) is established. The fact that there can 
only be one matrix X satisfying 

p(M) 

(i.e., that P is uniquely determined) can also be shown, but we shall omit the 
proof here because it is not particularly illuminating. 

Notice that (15.3-8) is essentially a set of simultaneous linear equations 
for the elements of P. Consequently, it is easy to solve. To illustrate the pro- 
cedure, we take the same example as before — viz., 




~0 0 " 




0 1 ~ 




~ 1 " 




'0 0“ 


J 


+ 


0 — v/T_ 


P~ P 


0 


[1 0 \P + 


_o L 



= 0 



with 



M 



'0 

1 

0 

0 



0 

-VT 

0 

-1 



1 

0 

0 

0 



0 * 
0 

-1 

VT 



The eigenvalues of M are 



— 1 twice and +1 twice. Accordingly, 
p(s) = s 2 + 2s + 1 



and 



p(M) = M 2 + 2 M + I 



i 


1 


0 


— 


2 


1 






~/2 -f 2 - 


-2^/2 


+ 3 - 


1 


0 






0 


1 




1 


- 


-VT 


- 2 




-1 


-2 




0 


2VT 


f 3_ 


Equation (15.3-8) becomes 














~ 1 


0 


-2 


1 


- 




' 1 


0 ~ 




VT + 2 


-2a/T + 3 


-1 


0 






0 


1 




0 


1 


I 


— VT — 


• 2 




Pi i 


Pi 2 




_-l 


-2 


0 


2VT f 


3_ 




—Pi 2 


#22_ 



= 0. 



This implies eight linear relations among the p ip only three of which are 
independent. Writing down only independent relations, we obtain 



1 - 2p n +p iZ = 0 
+2 — /? u = 0 
~2/7 l2 + P Zl = 0 
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whence 



"2 - VT 3 - 2a/T' 

3 - 2 VT 6 - 4a/T 



as earlier established. 

One characteristic of the two previous methods is that the eigenvalues of 
a certain matrix must be calculated. This is also a requirement in the method 
of the last section for computing the differential as distinct from the algebraic 
equation solution by diagonalizing a matrix to find its exponential. 

For large-order systems, eigenvalue computation may pose some prob- 
lems. Therefore, the next method to be discussed, presented in [10], may 
provide an acceptable alternative because no eigenvalue computation or 
polynomial factoring is required. The penalty is that the method recursively 
computes a sequence of estimates for the positive definite solution of (13.3-1), 
and, of course, in any given problem many iterations might be required. 

As usual, we consider the equation 



PF + F'P - PGR-'G'P + Q = 0 (15.3-1) 



and seek its positive definite solution. The first step in the procedure is to 
select a matrix K 0 such that F (- GK' 0 has all its eigenvalues with negative 
real parts. This is always possible if [F, G] is completely controllable (see 
[13] through [20]). Then a sequence P 0 , P t ,P 2 , . . . , of n X n matrices is 
defined in the following fashion. With 

F t = F+GK' (15.3-9) 

the matrix P t satisfies 

P,F, + F'P, + K t RK’ t + Q = 0. (15.3-10) 

Furthermore, 

K i+1 = -PfiR- 1 (15.3-11) 

which permits the carrying through of the next step of the iteration. 

It turns out that 

lim P, = P. (15.3-12) 

I— .oo 

Evidently, the computation procedure falls into two distinct phases. In the 
first phase, a matrix F 0 must be determined, such that F + GKq has all its 
eigenvalues with negative real parts. If the eigenvalues of F all have negative 
real parts, K 0 = 0 is an immediate acceptable selection. If not, more computa- 
tion is required. 

In the case when G is a vector rather than a matrix, though, the com- 
putation is straightforward. By first making a basis change if necessary, we 
can assume that F is in companion matrix form. That is, 
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and 



0 1 0 
0 0 1 





0 0 

“ a z -a 3 




1 



0 “ 
0 



i 



(15.3-13) 



(15.3-14) 



Then, if k f ~[k x k z ■ ■ • k n ], it is immediate that F + gk' is also a 
companion matrix, and the associated characteristic polynomial is s n + 
(a n - k n )s ■ | • • • — | — (a l — k { ). Choice of the k { to ensure that this poly- 
nomial has all its zeros with negative real parts is straightforward. In the 
case when G is not a vector but a matrix, selection of an appropriate K 0 
can be more involved (see [13] through [20]). 

In the second phase of the calculations leading to P , equations such as 
(15.3-10) are solved. Solution procedures for such equations are well known 
(see, e.g., [21]); the equation really is a rearranged version of a set of linear 
equations for the entries of the matrix P t . Therefore, aside from difficulties 
with high-order problems, calculation is straightforward. 

Notice that the end result is not unreasonable; if the sequence P 0 , P u 
. . . , approaches any limit at all, call it X , then K 0 , K u . . . , approaches 
XGR~ and taking limits in (15.3-10) results in 



X(F — GR-'G'X) + (F' - XGR~ l G')X + XGR~ l G'X + Q - 0 



or 



XF + F'X - XGR~ 1 G'X + Q = 0. 

To be sure, it is not clear that the sequence P 0 , P u . . . , should approach a 
limit, or, indeed, that this limit should be positive definite (and therefore equal 
to P). But, in fact, it can be shown [10] that 

A) > Pi > Pi > • • * > P 

from which it follows that the sequence does possess a positive definite limit. 
Reference [10] also shows that convergence of the sequence P 0 ,P l9 . . . , 
toward P is quadratic — i.e., 
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for some constant c and for all /. This guarantees rapidity of convergence 
when P t is close to F. A figure of 10 iterations is claimed as being typically 

required. 

A second iterative procedure under development [1 1] starts by replacing 
the continuous time minimization problem with a discrete time minimization 
problem. The replacement is such that solution of the discrete time minimiza- 
tion problem leads to a solution, including knowledge of P , of the continuous 
time problem. We now outline the procedure. 

Applying the transformations 

<5 = W - F')P(I - F) 

A — {I — F)~\I + F) 

B = 2(7 - F)~ 2 G 

C= R + G’(I- F'Y 1 Q(l F)~'G 
D = Q(I — F)~'G 

the quadratic matrix equation (15.3-1) may be written as 

A'QA - <5 - [A'OB + D][C + B'$B]-'[A’$B + £>']' + 0-0. 

It has been shown that the existence of a unique nonnegative definite solu- 
tion P of (15.3-1) implies the existence of a unique nonnegative definite solu- 
tion <5 of the preceding equation. This solution may be determined by solving 
a discrete Riccati equation as follows: 

O - lim 4>,. 

I — ► oo 

where 

<P /+1 =■ A'Q,A - [A'tyB + D][C + B’ibfiY'iA'ibtB + D] f + Q 

O 0 — 0. (15.3-15) 

It has been shown that the existence of a nonnegative definite solution of 
(15.3-1) implies convergence of this difference equation. Once O is calculated 
(convergence typically requires 10 iterations), then P may be determined 
from by the transformation 

P= 2 (/- F') _1 <5(/- F) 1 . 

We observe that at each iteration of the Riccati equation (15.3-15), 
the inverse of a matrix of dimension equal to the number of system inputs is 
required rather than the solution of a linear matrix equation of order equal 
to the state vector dimension, as in the previous method. Notice, also, that 
it is not required that an initial gain K be determined such that F + GK' 
has negative eigenvalues. Detailed comparison with other methods is being 
investigated at the time of this manuscript’s writing. 

Finally, in this section we wish to show how a transient Riccati differ- 
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ential equation solution may be found from a steady state solution. In other 
words, we wish to find an expression for the solution P{t) of 

-P = PF + FT - PGR 'GT + Q P( 0) - A (15.3-16) 

given a positive definite P satisfying the limiting version of (15.3-16): 

0 - PF + F'P - PGR-'G'P + Q. (15.3-1) 

The existence of an expression for P(t) in terms of P means that the 
techniques of this section for the calculation of P now have extended capabil- 
ity; they may be added to those of the previous section for the calculation of 
Pity 

With P the positive definite solution of (15.3-1) and 

F = F — GR-'G'P (15.3-17) 

and with the assumption that (A — P) -1 exists (which will certainly be the 
case if A = 0), we shall establish the following formula, valid for / < 0: 

P(t) = P + e- F '[e- F ‘Xe- F ‘ + (A - P)~' - X]~'e~ Ft (15.3-18) 

where X is the unique negative definite solution of 

FX + XF' — GR~ l G' = 0. (15.3-19) 

The significance of F may be recalled from our earlier discussion of the 
optimal regulator. The control law for the optimal regulator is u = 
— R~ ] G f Px, and therefore the closed-loop regulator is described by x = Fx . 
As we know, the closed-loop regulator is asymptotically stable; therefore, 
in (15.3-18), as t becomes more and more negative, the exponential terms 
decay. 

To establish (15.3-18), we start with the two equations (15.3-16) and 
(15.3-1). By subtraction, it follows that 

~4( p -P) = (P- P)F + F’iP — P) — (P — P)GR 'G'(P - P) 

- PGR-'G'P - PGR-'G'P + 2PGR-'G'P 
= (P — P)F + F'(P - P) - (P - P)GR 'G\P - P). 

Now assume that (P — P)" 1 exists for all /. Setting X(t) = [P(/) — P] _1 , it 
follows that X = ~(P — Py i [(d/dt)(P — P)](P — P) _1 , and, therefore, 

X = FX + XF' - GR'^G' (15.3-20) 

with 

X(0 ) = (15.3-21) 

[Note: Equation (15.3-20) always has a solution provided A"(0) exists. 
Consequently, the condition that [P(r) — P] _I exists is that (A — P)' 1 
exists.} 
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Now X satisfies the algebraic equation 

0 = FX + XF' - GR 'G'. # (15.3-19) 

It so happens that with F possessing all its eigenvalues with negative real 
parts, this equation— really a set of linear equations for the entries of X — 
always has a unique solution [21]. See also Appendix A. 

Now subtract (15.3-19) from (15.3-20) to get 

J-(X - X) = F(X - X) + (X - X)f 

with X — X having initial value (A — P) } — X. It follows immediately 
from this equation that 

X(t) - X = e Ft [(A - P)-' - X]e F \ 

Recalling now that X(t) is [P(/) — P]~\ we deduce 

P(t) P + [X + e Ft (A - Py i e Ft - e Ft Xe Ft ]~ l 

= P + e~ Fft [e Ft Xe- Ft + (A - P)~ l — X)~ x e~ Ft 

as required. 



Problem 15.3-1 . Using both methods discussed in this section, find the posi- 
tive definite solution of 



0 0 
1 



4- 



0 -1 

o VTJ 



i 

o 



[1 0 ]P 



Problem 15.3-2. Consider the equation 

PF + F'P - PGRr'G'P 4 6 0. 



Let 



M - 



and let 



r = 



F - GR X G ' 
Q -F' 



F ii F i 2 
LF 2l F 2 2_J 



be such that 



0 0 
_0 1 



0. 



FMF 



- 1 .. 



A 1 1 A 12 
-0 ^2 2—1 

Show that P = T 2 \ F[ ^ is a solution of the quadratic matrix equation. 



Problem 15.3-3. Using both methods of this section, find the positive definite 
solution of 



- 0 








~ 0 


/ 2 r 




~ 1 “ 




“i on 


_fl 1 


0 _ 


P 4 


P 


-/1 2 


0 _ 


- P 


_0_ 


[1 0]/ 5 4 


— 1 

0 

0 

r 



Assume / 12/21 ^ 0. Discuss also the cases /, 2 0 and f 2X 0. 
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Problem 15.3-4. Given the steady state solution 



of the equation 




_3 - 2*/l 



3 - 2a/2~~ 

6-4VX. 



'0 

_1 



0 “ 

V2". 



+ 



"0 

L0 



1 

-a/1 



p -p 



1 

0 J 



[1 o ]P + 



find an explicit expression for P{t) using the formula (15.3-18). 



'0 0 ‘ 
-0 1 _ 



Problem 15.3-5. Find an expression for the solution of 

— p = 2 pf — p 2 g 2 + h 2 
in terms of the positive solution of 

2pf~ P 2 g 2 4- h 2 = 0. 



15.4 APPROXIMATE SOLUTIONS OF HIGH-ORDER 

RICCATI EQUATIONS VIA SINGULAR PERTURBATION 



The computing time required to obtain a solution to high-order Riccati 
equations may be too long for some applications, and since the computing 
time increases at least according to the square of the system order, there is 
the temptation to neglect second-order effects, which increase the dimension 
of the state vector. This temptation is particularly strong when many solu- 
tions of a Riccati equation are required. However, experience shows that to 
carry out a design neglecting second-order effects altogether often leads to 
completely erroneous results. 

Recent studies [22] and [23] have shown that the Riccati equation 
solutions obtained by neglecting the second-order effects, at least in two major 
areas of application, are not irrelevant to a problem solution. The addition 
of further relatively straightforward calculations enables an approximate 
solution to the high-order Riccati equation to be determined from the exact 
solution to the low-order equation or equations. For many design problems 
that occur in practice, this approach to obtaining a Riccati equation solution 
is quite adequate and considerable computing time can be saved. 

We shall consider two cases that arise in control applications, where 
an tfth order system (not necessarily time invariant) 

x = Fx + Gu (15.4-1) 

is to be controlled to minimize an index 

V(x(t 0 \ «(•)> T) = f (u'u + x'Qx) dt (15.4-2) 

J to 

where Q is nonnegative definite symmetric. (Without loss of generality, we 
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have assumed for simplicity that the usual u Ru term is simply u'u.) The Ric- 
cati equation of interest is 

-P - PF + F'P ~ PGG'P + Q P(T,T) = 0. (15.4-3) 



The case of nominally “decoupled” systems. The first case considered 
is the case where the second-order effect consists of coupling between two 
nominally decoupled systems. That is, the equations for system (15.4-1) 
may be partitioned as follows: 



-*r 

where 6 is a scalar parameter, assumed small. If e is set to zero, then (15.4-4) 
is the equation of two decoupled systems. The matrix Q in the performance 
index may be partitioned as 



^ €F X 2 
F 2 



+ 



LW2 



Gl ^12 

G r 



u { 

u, I 



(15.4-5) 



Qx *Q 12 

CQ\ 2 Qi J 

and the associated Riccati equation solution P as 



(15.4-5) 



P\ 



P 12 

P. 



(15.4-6) 



12 1 2 

We now adopt the notation P = P | (e = 0), and P t = | ( dP/de ) (6 = 0). 
Differential equation theory yields that P is analytic in e, and therefore P 
may be expanded as a Maclaurin series expansion as follows 

P = P + eP ( + Higher-order terms. (15.4-7) 

Straightforward manipulations (the derivation of which is called for in 
Problem 15.4-1) yield that 



(15.4-8) 



p 1 


0 ~ 


p, = 


"0 


F I2f~ 


^0 


A_ 




p' 

L r 1 2f 


0 



where 

~A = P t F l + F\P , - P l G i G\P l + Q ? { (T T) = 0 
-A = P 2 F 2 + f’ 2 P 2 - P 2 g 2 g' 2 P 2 + q 2 P 2 (t, t) = o 
-A 2 ,(^ - G 2 C'A) + (f, - G^P,)'?^ + y 

p I 2 ,(r, r> = o 

and 



(15.4-9) 

(15.4-10) 

(15.4-11) 



Y - /\F 12 + F 21 F 2 - P,(G { G 1X + G i2 G' 2 )P 2 + 0 12 . (15.4-12) 

When the nominally decoupled systems are assumed totally decoupled, 
the parameter e is taken to be zero and P is approximated by P. The matrix 
P is determined from two Riccati equations, (15.4-9) and (15.4-10), each of 
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lower order than (15.4-3). There are many situations where this approach 
results in unsatisfactory system performance — such as, e.g., where tempera- 
ture, pressure, and flow control systems are designed independently despite 
the coupling that occurs between them in a chemical process [22]. 

For the case when coupling is small, the parameter e is small, and an 
improved approximation for P is given from P ~ P + eP e . To calculate P e 
simply requires the solution of a linear equation (15.4-11) and (15.4-12). 
Higher-order terms in the Maclaurin series expansion may be taken into 
account without much further effort, as indicated in reference [22]. 

The corresponding time-invariant infinite-time problem is easy to solve. 
The limiting solutions of (15.4-9) and (15.4-10) exist if [F l5 <7,] and [ F 2 , G 2 ] 
are completely controllable, respectively. The limiting solution of (15.4-11) 
will exist if the eigenvalues of F t — G { G\P { and F 2 — G 2 G' 2 P 2 all have nega- 
tive real parts. As we know, sufficient conditions guaranteeing that such will 
be the case are that [F u D x ] and [F 2 , D 2 \ are completely observable for any 
D x and D 2 with D l D\ = Q x and D 2 D' 2 = Q 2 . These various conditions all 
have obvious physical significance. 

In summary, then, an approximate method for solving Riccati equations 
associated with nominally decoupled systems has been given, which requires 
the solution of low-order Riccati equations and linear equations rather than 
the solution of a single high-order Riccati equation. For large-order systems, 
this represents a considerable saving in computer time. Examples and per- 
formance analysis details are given in [22]. 

The case of a nominally low-order system. The second solution we 
examine arises when the system under consideration is exactly described by 
high-order equations, but approximately described by low-order equations. 
The approximation is possible because some time constants are extremely 
fast, and their effect on the system’s dynamics is almost negligible. In pole- 
zero terms for time-invariant systems, some of the poles of the system transfer 
function may be well in the left half-plane, with their residues in a partial 
fraction expansion being very small; the approximation would involve 
neglecting these poles. 

We assume that the system equations (15.4-1) are the nominal or approx- 
imating low-order equations and that the performance index (15.4-2) is an 
index associated with this nominal low-order system. The Riccati equation 
(15.4-3) enables an optimal control to be obtained for the low-order system. 

Now experience shows that when this control computed by neglecting 
the small time constants is applied to the actual physical system, unsatisfac- 
tory (perhaps unstable) performance often results [23]. To avoid this situation, 
the small time constants must be taken into account in some way. One way 
to do so is to avoid any approximations at all by using an exact high-order 
system description and by solving a high-order Riccati equation. However, 
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we shall steer a path intermediate between the exact analysis and an analysis 
involving total neglect of the fast time constants. This results in some com* 
putational saving while normally retaining satisfactory closed-loop perfor- 
mance. 

The approach adopted for this problem is essentially the same as that 
adopted for the case of nominally coupled systems. Namely, we introduce 
a small parameter, now denoted by A, to expand the solution of the Riccati 
equation associated with the high-order system as a Maclaurin series in A, 
and we approximate this solution by taking into account the first few terms 
of the series expansion. The calculations involved in this approach require 
less computer time than if the high-order Riccati equation is solved directly. 

As a first step, we express all the small time constants in terms of a para- 
meter A, where it is understood that A has a known small value for the given 
system. For example, if a time constant t has a value t = 0.04, it is written 
as t — 0.4A (say), where it is understood that A has the value A — 0.1 for the 
given system. 

When the states associated with the small time constants are denoted by 
z, the high-order system description turns out to have the form 



x 

Az 



~F X 




(15.4-13) 



In this equation, we stress again that A is small compared to unity. Further- 
more, the entries of the F t and (7,., although possibly dependent on A, have 
comparable magnitudes. In our derivation here, we shall assume that the F ( 
and G t are independent of A, as is often the case. Problem 15.4-4 asks for this 
restriction to be removed. 

Taking A equal to zero amounts to neglecting the fast time constants. 
Therefore, by setting A 0 in (15.4-13), we should be able to recover the 
low-order system equation (15.4-1). The mechanics of this operation proceed 
formally as follows. Assuming invertibility of F 4 , 



0 = F 3 x + F 4 z + G 2 u 



or 

z = -F 4 \F 3 x + G 2 u) (15.4-14) 

and from (15.4-13), again, 

x - - F 2 F 4 l F 3 )x + (Gj - F 2 F; l G 2 )u. (15.4-15) 

The qualification that (15.4-13) and the pair (15.4-14) and (15.4-15) are only 
formally equivalent comes about in the following way. What is desired is 
that the solutions [x(t. A), z(/, A)] of (15.4-13) should approach the solutions 
[*(/), z(/)] of (15.4-14) and (15.4-15) as A approaches zero. Reference [24] 
shows that this occurs for t > t Q if and only if the eigenvalues of F 4 have nega- 
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tive real parts for all t. It may be thought that this is an unreasonable restric- 
tion to make, but, in actual fact, it is not, since it amounts to insisting (in 
the constant F 4 case) that all poles that we wish to neglect lie in the half- 
plane R c[s] < 0. 

Equation (15.4-15) agrees with Eq. (15.4-1) if we put 



F — F { — F 2 F;'F 3 G - G { - F 2 F 4 ] G 2 . (15.4-16) 

Equation (15.4-16) may be regarded as a set of relations guaranteeing that 
(15.4-1) and (15.4-13) be compatible representations of the same system. 
They also provide a formal procedure for passing from the exact, high-order 
representation of the system to the nominal, approximate, low-order repre- 
sentation. 

The performance index associated with the high-order system equations 
will be denoted by 

K(*(/ 0 ), z(/ 0 ), *(-), T) - f (uu + xQx) dt (15.4-17) 

J to 

where Q is nonnegative definite. Subsequently, we shall consider the time- 
invariant infinite-time problem. It is possible to take as a performance index 
for the high-order system an index where the loss function is of the form 



u'u + [x f 




as shown in [25], However, there is in general no need to include z(*) in the 
performance index, since the exponential rate of decay of z(t) is assumed high 
a priori without any feedback control. 

The solution of the Riccati equation associated with the high-order sys- 
tem (15.4-13) and the preceding index (15.4-17) will be denoted by IT. The 
matrix II may be regarded as a function of A as well as of t — i.e., each A gives 
a different value of II. In terms of a partition of II as 




AH 2 

An 3 _ 



(15.4-18) 



and the use of the superscript tilde to denote evaluation at A = 0 and the 
subscript A to denote differentiation with respect to A, our aims will be as 
follows : 

1. To observe that the matrices IT, are continuous at A — 0, and that the 
II, satisfy an equation derived from that satisfied by II by formally 
setting A = 0 in this latter equation. 

2. To show by using the equations derived for the II, that these matrices 
may be computed in terms of the solution P of the equation (15.4-3), 
associated with the low-order approximation to the control problem. 
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3. To observe that the matrices II,. are continuously differentiable at 
X — 0, and that the matrices rt iA satisfy an equation derived from that 
satisfied by U x by formally setting X = 0 in this latter equation. 

4. To observe that the matrices ff a may be derived by the solving of a 
linear matrix differential equation, and the carrying out of simple 
algebraic manipulations. 

The conclusion is that II may be approximated by 



Hi + Aftu A(fi 2 + xu 2X y 
A(iT 2 + xn f 2x ) X(fi 3 + An 3A )_ 



(15.4-19) 



provided X is small, with the approximate value of II far easier to compute 
than the exact value. The optimal feedback gain K for the high-order systems 
(15.4-13), normally given by 



K 




(15.4-20) 



is instead approximated to first order in X by 

r n 1 g 1 + xn xl Gi + n 2 ^2 + ^ 1 x g~ 
" Uft ' 2 g, + n 3 G 2 + xu 3X g 2 



(15.4-21) 



We now deal with each of the preceding aims 1 through 4, in turn. 
Using the high-order system equation (15.4-13), the loss function in (15.4-17), 
and the partitioning of II in (15.4-18), we have 



n, 

AITz 




= 

xn } _ 

-}- r, rr 2 
+ nn' 



n,F, + u 2 f } 

Air 2 F, + II3F3 

xf \ n 2 4 f \ rr 3 

XF 21^2 H F 4IT3J 



n,F 2 + n 2 F 4 

An' 2 F 2 + n 3 F 4 

IljGj 4 - n 2 G 2 
_ _xn ' 2 g, + n 3 G 2 



x [< 7 ,n, + c' 2 n ' 2 



xg\ n 2 + G 2 n 3 ] + 




0" 

0 



(15.4.22) 



with n,(F) = 0 for i = 1,2, 3. 

Standard differential equation theory guarantees that the solution 
of this equation depends analytically on X away from X = 0. But because 
setting X — 0 amounts to lowering the order of the differential equation — as 
is seen from examination of the left side of (15.4-22) — special techniques must 
be used to conclude even continuity at X = 0. A theorem of [24], however, 
turns out to be applicable. Precisely because of the earlier condition imposed 
on F„, it is possible to conclude that (1) the matrices IT,- are continuous at 
X = 0 for all t < T, and that (2) the matrices n,(0 satisfy the equations 
obtained by formally setting X = 0 in (15.4-22): 
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L 



U 



T 



~n, o 

dt 1 
0 0 

F\ft x + f\ n; nn 3 

f\ n, + nm nn 3 j 



+ n 2 F 3 0^2 + n 2 F 4 

n 3 F 3 n 3 F 4 



+ h 2 g 



n 3 G 2 



x [G' n 1 + G' 2 fr 2 G' 2 n 3 ] + 



Q o- 

0 0 



(15.4-23) 



together with IT X {T) — 0. This achieves aim 1. 

We now set out to achieve aim 2 — i.e., to use the equations (15.4-23) 
to derive formulas for the matrices II, in terms of P, the solution of (15.4-3). 
First, we consider the 2-2 block of (15.4-23), which yields 



ii 3 f 4 + nn 3 = n 3 G 2 G , 2 n 3 . 



Now this equation can have a multiplicity of solutions, including, obviously, 
the zero solution. However, it is not difficult to show, using the eigenvalue 
constraint on F 4 , that all solutions are nonpositive definite. Now for X non- 
zero, the matrix IT is nonnegative definite for all /, and therefore so is the 
submatrix 2XI 3 . Hence, n 3 is nonnegative for all 2^0, and since n 3 is 
the continuous limit as X approaches zero of fi 3 , ft 3 must be nonnegative 
definite. Consequently, fl 3 must be simultaneously nonpositive and nonnega- 
tive definite. The only possibility is 

fl 3 = 0. (15.4-24) 

Examination of the block in the 1-2 position of (15.4-23) yields that 

fl 2 - (15.4-25) 

When this expression for II 2 is inserted into the block in the 1-1 position of 
(15.4-23), there results the following differential equation for Hj : 

= ff,(F, - F 2 FVF } ) + (F l - F 2 F; 1 F 3 )'fi 1 
- - F 2 F 4 -*G 2 )(C, - FtFi'GJ fi, + Q. 

On using the definitions (15.4-16) for Fand G, we observe that this equa- 
tion becomes identical with the differential equation (15.4-3) for P. The 
boundary conditions also match. Hence, 

n x = P. (15.4-26) 

Equations (15.4-24) through (15.4-26) fulfill the requirements set out in 
aim 2. 

We pass on now to aim 3, requiring observation of the differentiability 
at X = 0 of the matrices TI„ and the derivation of equations satisfied by the 
matrices U u . As remarked earlier, solutions of the Riccati equation (15.4-22) 
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are analytic away from A - 0. In particular, we can differentiate (15.4-22) to 
obtain 



ir 1A ri 2 i ait 2A 
iTz t ^ri 2A ri 3 1 m n _ 

n u F t ! n 2A F 3 u [X f 2 h 22 f 4 



_n ; f, i 


2n ' 2 a f, 


I n u f> 


n ' 2 f 2 i 


*rr M F 2 -i 


F\n ix 


! X n' A 


f\u 2 i- 


if\ n Zi 


i- f\ n u - 
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AF' 2 n 22 
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ro 
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^ •* 
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uC, ! n 


ifi i 






_n ' 2 Ci i 


2n 2- ,g , 


4 n 3i c 2 







x [g; n, 



gjl AG;n 2 1 G' 2 n 3 ] 



" n^, i n 2 G 2 
_An'G r 1 n 3 G 2 



x [G\n u i g' 2 it 2a g\u 2 i ag;ii 2A -i G 2 n 3A ] (i 5.4-27) 

with rur) 0 for / = 1, 2, 3. 

Once again, the results of [24] combine with the properties of F 4 to yield 
the conclusion that the matrices IT iA are continuous at A 0, and that equa- 
tions satisfied by the matrices II lA may be obtained by formally setting A - 0 
in (15.4-27). These equations are [with the matrix X defined in (15.4-31)] 



a T I 

dt llu " 


I~UF, G,(G',F { C' 2 II' 2 )] 






| [F, - G,(G',F 4 C' 2 lT z )]'fl u 1 


X 


(15.4-28) 


n 2i - 


n 2 - fl u F 2 - F\ fi 2 








4 ( PG , i n 2 G 2 )(G',ri 2 -1 G'IT„) 


F ' 

1 4 


(15.4-29) 


and 








0 - 


i~i„f 4 + f; ft 32 + (F' 2 fl 2 + IT 2 F 2 ) 




(15.4-30) 


with the initial condition fi lA (r) - 0. In equation (15.4-28), the quantity X 



is given by 



X ----- + FJV 



2H 3 I 1 3 AA 2Z (PG\ + U 2 G 2 )G' 2 IV 22 

n Z ,G 2 (G\P + G 2 n 2 ) 



(15.4-31) 



The derivation of these equations means that aim 3 is achieved. Achieve- 
ment of aim 4 is immediate: Eq. (15.4-30) is a readily solvable linear matrix 
equation yielding II 3A . This then allows elimination of II 2A from (15.4-28) 
by using (1 5.4-29) ; then equation ( 1 5.4-28) becomes a linear matrix differential 
equation in II IA , and is readily solved. 

The time-invariant problem is also straightforward to solve. The matrices 
F h G„ and Q are, of course, constant, and T = +oo. One then seeks limiting 
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solutions of the Riccati equation for P and a limiting solution of the linear 
differential equation (15.4-28) for n u . Since this equation in its limiting form 
is a linear matrix equation, it is probably easier to determine its solution by 
solving the algebraic equation obtained by setting (d/dt) II u equal to zero, 
than by obtaining the limiting differential equation solution. Let us now sum- 
marize the calculations required. 



1. We start with a high-order system as in (15.4-13), with the F t inde- 
pendent of X. {If the F t depend continuously on X at X = 0, we must 
use more complex formulas the development of which is called for 
in Problem 15.4-4, or which may be obtained from [23].} We aim 
to minimize the performance index (15.4-17). 

2. We compute the equations of the nominal low-order system (15.4-1) 
using (15.4-16) and the matrix P of Eq. (15.4-3). 

3. We compute the matrices IT, via (15.4-24) through (15.4-26). 

4. We compute the matrices ll a via (15.4-28) through (15.4-31). 

5. The matrix II is approximated using the approximate values for the 
matrices IT/2) in (15.4-19); the approximate value of II is used to 
determine an approximate control law in (15.4-21). 



Reference [23] includes some numerical examples, which compare exact 
designs with approximate designs obtained by taking X = 0, and by using the 
matrices ft, and n /A as discussed herein. Better approximation to the exact 
design could no doubt be obtained in these examples, and indeed in all situa- 
tions, by computing fl iU and higher derivatives. The computational task, how- 
ever, will probably approach that associated with an exact design and negate 
the utility of such an approach. 

We now present a simple example illustrating use of the preceding ideas. 
Although the example does not demonstrate large savings in computation 
(because in this case the “high-order’’ system is not really high order), it does 
indicate the sequence of calculations required. 

The system we consider, shown in Fig. 15.4-1, is 




which we choose to write as 




x 

z 






0 

5 



u 








u 




Fig. 15.4-1 The high-order system. 
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where X = 0.1. The performance index we shall take to be 




The first requirement is to obtain the associated nominally low-order system. 
Straightforward application of (15.4-16) yields this to be 



x = x + \u. 




Fig. 15.4-2 The approximate low- 
order system. 



Next, we solve the optimization problem associated with this low-order 
system. The optimal performance index is Px 2 , where P is the positive definite 
root of 



IP - \P 2 + 1 = 0 




P= 4 + 2a/5\ 

Applying (15.4-24) through (15.4-26), we obtain 

n l = 4 + 2VT, n 2 = 4 + 2VX n 3 - o. 

Equations (15.4-28) through (15.4-31) are now used to obtain the matrices 
n a . First, from Eq. (15.4-30), we have 

n 3A -(4 + 2VT). 



Equation (15.4-29) yields 

n u = II U + (5 +2VT). 

Substituting into the steady state version of (15.4-28), we obtain 

fl 1A - -9 - 4 v / T. 



Thus, 



IL, — — 4 — 2a/^5~. 



From Eq. (15.4-21), the approximate feedback gain vector is 



K = - 



"(2 + VT) - X(2 4 «/Ty 
X(2 + VT) 



The difference is clear between taking 2 = 0 (corresponding to total neglect 
of the dynamics associated with z) and X = 0.1, (corresponding to an 
approximation consideration of the dynamics associated with z). Problem 
15.4-5 asks for a comparison with the exact feedback gain. 
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Problem 15.4-1. Using Eqs. (15.4-3) through (15.4-6) of the text, derive 
Eqs. (15.4-9) through (15.4-12) for P and P t . 



Problem 15.4-2. Derive recursive formulas for 



and 



d 2i P 

de 2i 



(* = 0 ) 



d 2i+l P 
de li+ 1 





given Eqs. (15.4-3) through (15.4-6). 



Problem 15.4-3. Given a plant with a transfer function 
iV(s) 3 3 6 32 



25 



OAs + 1 0.045- + 1 0.07.y -f 1 2s + 1 5s + 1 

express the state-space equation for the system in the form of (15.4-13). 



Problem 15.4-4. Assume that the matrices F, in Eq. (15.4-13) depend continu- 
ously on X in the vicinity of X = 0. Indicate the variations necessary in the subse- 
quent derivation of the formulas for 



Problem 15.4-5. Compare the approximate feedback gain and performance 
index obtained in the worked example with the exact feedback gain and perform- 
ance index. 



Problem 1 5.4-6. Can you suggest how the results of this section can be applied 
to optimal filtering problems? 
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APPENDIX 



BRIEF REVIEW OF SOME 
RESULTS OF MATRIX 

THEORY 

The purpose of this appendix is to provide a rapid statement of those 
particular results of matrix theory used in this book. For more extensive 
treatments standard text books — e.g., [1] and [2] — should be consulted. The 
latter is a particularly useful source of computational techniques. 

1. Matrices and vectors. An m x n matrix A consists of a collection of 
mn quantitiest a ij (i — 1 , 2, . . . , m; j = 1, 2, ... , n) written in an array of 
m rows and n columns: 

a \\ a \2 

® 2 \ &22 

A = * 

• « 

• • 

^m2 * * * 

Sometimes, one simply writes 

A = (a,.,). 

The quantity a, 7 is an entry (the (i-y)th entry, in fact) of A. 
tThe au will be assumed real in most of our discussions. 
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An m vector , or, more fully, a column m vector , is a matrix with 1 column 
and m rows; thus. 



*1 

*2 




defines x as column m vector, whose ith entry is the quantity x { . A row n 
vector is a matrix with 1 row and n columns. 



2. Addition, subtraction, and multiplication by a scalar. Two matrices 
A and B with the same number of rows and also the same number of columns 
may be added, subtracted, or individually multiplied by a scalar. With k x , 
k 2 , scalar, the matrix 

C = k x A + k z B 

is defined by 




k\ a ij + k 2 




Thus, to add two matrices, one simply adds corresponding entries; to sub- 
tract two matrices, one simply subtracts corresponding entries; etc. Of course, 
addition is commutative — i.e., 

A T B — B -T A. 



3. Multiplication of matrices. Consider two matrices A and B , with A 
an m x p matrix and B a p x n matrix. Thus, the number of columns of A 
equals the number of rows of B. The product AB is an m x n matrix defined 
by 

C — AB 

with 

p 

C ij = £ a ik^kj' 
k = i 

Notice that C has the same number of rows as A , and the same number of 
columns as B . 

The product of three (or more) matrices can be defined by 

D = ABC ~ (AB)C - A(BC). 

In other words, multiplication is associative. However, multiplication is not 
commutative — i.e. it is not in general true that 



AB = BA. 
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In fact, although AB can be formed, the product BA may not be capable of 
being formed. 

For any integer p, the p X p matrix 



'1 0 0 ' 

0 1 0 



0 0 -.. 1 



possessing p rows and columns is termed the identity matrix of order p. It 
has the property that with A any m x p matrix, 



AI= A. 



Likewise, the identity matrix of order m has the property that 

IA = A. 



Any matrix consisting entirely of entries that are zero is termed the zero 
matrix . Its product with any matrix produces the zero matrix, whereas if it 
is added to any matrix, it leaves that matrix unaltered. 

Suppose A and B are both n x n matrices (A and B are then termed 
square matrices). Then AB is square. It can be proved then that 



\AB\ 



A\\B 



where \A ] is the determinant of A. 

[The definition of the determinant of a square matrix is standard. One 
way of recursively defining \A \ for A an n x n matrix is to expand A by its 
first row, thus 





#2 2 

* 


« • * 


#2 » 

* 




#21 

#31 


a 2 3 

#33 


#24 

#34 


* • * #2/1 

’ • ‘ #3 * 


11 


• 




• 

• 


#12 


* 

* 






• 

• 




#n2 


• • ♦ 


#/JM 




• 

#n 1 


#n3 


#n4 


• 

• • * a 

“nn 



#2 1 #22 #24 * * * #2/i 

#3 1 #32 #34 * ‘ * #3 n 



+ #13 



#/Jl #/»2 #/j4 



a 



nn 



This expresses \A \ in terms of determinants of (n — 1) x (n — 1) matrices. 
In turn, these determinants may be expressed using determinants of (n — 2) 
X (n — 2) matrices, etc.] 
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4. Direct sum of two matrices. Let A be an n X n matrix and B an 
« 

m X m matrix. The direct sum of A and B, written A -f B, is the (n + m) 
X (n + m) matrix 

A 0 “ 

0 s 



where 



Thus, if 



Suppose A 


is 


an 


m 


X m matrix defined bj 


B - 


= A 1 




b, 




- a , 






j 


j 1 




ri 


3 


2~ 


A — 










L2 


I 


5_ 


i 


“1 


r 




A 1 -] 

i 


3 


i j 


« 




2 


5] 





It is easy to establish the important result 

(A By B'A', 

which extends to 



(ABC)' -= C B'A' 
and so on. Also, trivially, one has 

(A -f B)' -•= A' + B’ 



6. Singularity and nonsingularity. Suppose A is an n X n matrix. Then 
A is said to be singular if \A \ is zero. Otherwise, A is termed nonsingular. 



7. Rank of a matrix. Let A be an m x n matrix. The rank of A is a posi- 
tive integer q such that some q X q submatrix of A, formed by deleting (m — q) 
rows and (n — q) columns, is nonsingular, whereas no (q + 1) x (q + 1) 
submatrix is nonsingular. For example, consider 



A 



~ 1 2 3 CF 

J 2 3 0_’ 



The maximum size square submatrix that can be formed is 2 x 2. Therefore, 
a priori, rank A < 2. Now the possible 2x2 submatrices are 



'1 T 




“1 3" 




‘1 0 


r 2 


T 




~2 


(T 




"3 


(T 


1 2 




1 3 




J 0 


l_2 


3 




_2 


0 




3 


0 
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These all have zero determinant. Therefore, rank A < 2. Of the 1 X 1 
submatrices, two have zero determinant but six do not. Therefore, rank 
A = 1. 

The rank of A is also the maximum number of linearly independent 
rows of A and the maximum number of linearly independent columns of 
A. In the example, the second row equals the first row. Furthermore, the 
second, third, and fourth columns are linear multiples of the first. 

It can be shown that 

rank (AB) < min [rank A , rank B] 

If rank A is equal to the number of columns or the number of rows of 
A, A is often said to have full rank. If A is n x n, the statement rank A = n 
is equivalent to the statement A is nonsingular. If, for an arbitrary matrix 
A, rank ^4 = 0, then A is the zero matrix. 



8. Range space and null space of a matrix. Let A be an m x n matrix. 
The range space of A, written 3%[A\, is the set of all vectors Ax } where x 
ranges over the set of all n vectors. The range space has dimension equal to 
the rank of A — i.e., the maximal number of linearly independent vectors in 
3$[A\ is rank A. The nullspace of A , written ,A r [A\ is the set of vectors y for 
which Ay = 0. 

An easily proved property is that 3t[A r ] and A r [A] are orthogonal — i.e., 
if y x = A'x for some x, and if y 2 is such that Ay 2 = 0, then y\y 2 ~ 0. 



9. Inverse of a square nonsingular matrix. Let A be a square matrix. 
If, but only if, A is nonsingular, there exists a unique matrix, call it B , termed 
the inverse of A , with the properties 

BA = AB = /. 



The inverse of A is generally written A~\ There are many computational 
procedures for passing from a prescribed A to its inverse A~ l . A formula 
is, in fact, available for the entries of B = A' 1 , obtainable as follows. 

Define the cofactor of the i — j entry of A as (— 1) ,+/ times the deter- 
minant of the matrix obtained by deleting from A the /th row and yth column, 
i.e., the row and column containing a ij . Then, 



It easily follows that 




1 

A 



x cofactor of a ... 



(A~ i y = 

If A, and A 2 are two n X n nonsingular matrices, it can be shown that 



(A,A 2 )-' =A;'A r‘. 
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10. Powers of a square matrix. For positive m 9 A m for a square matrix 
A is defined as AA • • • A, there being m terms in the product. For negative 
m, let m = — w, where n is positive; then A m ~ ( A~ v ) n . It follows that A p A q 
= A p+q for any integers p and q , positive or negative, and likewise that 
{A p Y = A pq . 

r 

A polynomial in A is a matrix p(A ) = £ a.A 1 where the < 7 , are scalars. 

f=0 

Any two polynomials in the same matrix commute — i.e., p(A)q(A) == 
q(A)p(A), where p and q are polynomials. It follows that p(A)q~ l (A) = 
q~ l (A)p(A ), and that such rational functions of A also commute. 



11. Exponential of a square matrix. Let A be a square matrix. Then 
it can be shown that the series 

I + A + -jy A 2 

converges, in the sense that the i-j entry of the partial sums of the series con- 
verges for all i and j. The sum is defined as e A . It follows that 

e At = / +At + yj A 2 t 2 + • • • 

Other properties are :p(A)e At = e At p(A) for any polynomial A, ande~ At = [e At ]~ x . 



12, Differentiation and integration. Suppose A is a function of a scalar 
variable t, in the sense that each entry of A is a function of t. Then 



dA / da u \ 
dt \ dt ) 



It follows that 



-f(AB) = ~B + A~ 
dt ’ dt dt 



Also, from the definition of e A \ one has 

d 



dt 



(e At ) ~ Ae At = e At A. 



The integral of a matrix is defined in a straightforward way as 

J A dt = ^ J a ij dt^J. 

Suppose 0 is a scalar function of a vector x. Then 

^ = a vector whose /th entry is 

UA 1/ 

Suppose <j> is a scalar function of a matrix A. Then, 

= a matrix whose if entry is ^ - * 
dA da,, 
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Suppose z is a vector function of a vector x. Then, 



dz_ 

dx 



A matrix whose i-j entry 




13. Eigenvalues and eigenvectors of a square matrix. Let A be an n x n 
matrix. Construct the polynomial \sl — A\> This is termed the characteristic 
polynomial of A; the zeros of this polynomial are the eigenvalues of A. If 
X t is an eigenvalue of A, there always exists at least one vector x satisfying 
the equation 

Ax = A { x. 

The vector x is termed an eigenvector of the matrix A . If X t is not a repeated 
eigenvalue — i.e., if it is a simple zero of the characteristic polynomial, to 
within a scalar multiple x is unique. If not, there may be more than one 
eigenvector associated with X r If X t is real, the entries of x are real, whereas 
if X; is complex, the entries of x are complex. 

If A has zero entries everywhere off the main diagonal — i.e., if a i} ■=■ 0 
for all /, /, with i j, then A is termed diagonal (Note: Zero entries are still 
permitted on the main diagonal.) It follows trivially from the definition of an 
eigenvalue that the diagonal entries of the diagonal A are precisely the eigen- 
values of A. 

It is also true that for a general A , 

MI = fU- 

j=1 

If A is singular, A possesses at least one zero eigenvalue. 

The eigenvalues of a rational function r(A) of A are the numbers r(A / ), 
where X { are the eigenvalues of A. The eigenvalues of e At are e Xtt . 

14. Trace of a square matrix A. Let A be n x n. Then the trace of A , 
written tr[^4], is defined as 

tr[/l] = 2 a it . 

i= 1 

An important property is that 

tr [A] = ± A, 

i=l 

where the X t are eigenvalues of A. Other properties are 

tr[^4 + B] = tr [B + A] = tr[^4] + tr[5] 

and, assuming the multiplications can be performed to yield square product 
matrices, 

ir[AB] - tr [B'A'} = tr [BA] = tr [A'B'] 
tr [A 9 A] = ££ a?j. 

i= 1 J = 1 
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15. Orthogonal, symmetric, and skew-symmetric matrices, and their 
eigenvalue properties. If a square A is such that AA' = /, and thus A' A = I , 
A is termed orthogonal. The eigenvalues of A then have a magnitude of unity. 
If >4 = A', A is termed symmetric , and the eigenvalues of A are all real. More- 
over, if x x is an eigenvector associated with x 2 with A 2 , and if X x ^ X 2 , 
then x\x 2 = 0. The vectors x x and x 2 are termed orthogonal. {Note: Distin- 
guish between an orthogonal matrix and an orthogonal pair of vectors.) 
If A - -A', A is termed skew, or skew symmetric , and the eigenvalues of 
A are pure imaginary. 

16. The Cayley-Hamilton theorem. Let A be a square matrix, and let 
\sl — A | = 5" + a^”' 1 + • • • + a n ; then, 

A n -J- oc x A n 1 T - * * * T - ot n I — 0. 

The Cayley-Hamilton theorem is often stated, rather ambiguously, as 
“the matrix A satisfies its characteristic polynomial.” 

From the Cayley-Hamilton theorem, it follows that A m for any m > n 
and e A are expressible as a linear combination of /, A, . . . , A n ~ l . 

17. Similar matrices and diagonalizability. Let and B be X n matrices. 
If there exists a nonsingular n X n matrix T such that B = T~ l AT, the matrices 
A and B are termed similar. Similarity is an equivalence relation. Thus: 

1. A is similar to A . 

2. If A is similar to B , then B is similar to A. 

3. If A is similar to B and B is similar to C, then A is similar to C. 

Similar matrices have the same eigenvalues. This may be verified by 
observing that 

§ 

sl~ B= T~ l sIT — T~ X AT = T~ l {sl ~ A)T. 

Therefore, 

\sl- B\ = IT- 1 ])*/ - A\\T\ = |j/-i4||r- 1 ||r|. 

But T' X T = / so that | T~ x \ | T\ = 1. The result is then immediate. 

If, for a given A , a matrix T can be formed such that 

A - T~'AT 

is diagonal, then A is termed diagonalizable , the diagonal entries of A are 
eigenvalues of A , and the columns of T turn out to be eigenvalues of A. 

Not all square matrices are diagonalizable, but matrices that have no 
repeated eigenvalues are diagonalizable, as are orthogonal, symmetric, and 
skew-symmetric matrices. 

18. Jordan form. Not all square matrices are diagonalizable. But it is 
always possible to get very close to diagonal matrix via a similarity trans- 
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formation. In fact, there always exists a matrix T such that 

a, i 

A, 

A, 

A 2 1 



T-'AT - 



A 3 1 



a 3 1 



or something similar. Here, all blank entries are zero, the eigenvalues of A 
occur on the main diagonal, and there may or may not be entries of 1 above 
and to the right of repeated eigenvalues — i.e., on the superdiagonal. For any 
A , the distribution of l’s and 0’s on the superdiagonal is fixed, but different 
A yield different distributions. The preceding almost-diagonal matrix is 
called the Jordan canonical form of A. The Jordan blocks of A are the matrices 



'© ^ 
1 1 




"A 2 r 

0 A 2 




"A 3 1 0" 

0 A 3 1 




L— Z. 




o 

0 

N- 

CO 

1 



19. Positive and nonnegative definite matrices. Suppose A is n X n and 

symmetric. Then A is termed positive definite, if for all nonzero vectors x 
the scalar quantity x' Ax is positive. Also, A is termed nonnegative definite 
if x'Ax is simply nonnegative for all nonzero x. Negative definite and non- 
positive definite are defined similarly. The quantity x'Ax is termed a quadratic 
form , because when written as 

n 

x'Ax = QijXiXj 

i,j= 1 

it is quadratic in the entries x i of x:. 

There are simple tests for positive and nonnegative definiteness. For A 
to be positive definite, all leading minors must be positive — i.e., 



a lt > 0 



a 



a 



1 1 



12 



a 



12 



>0 



a 



a 



1 1 



a 



12 a 22 u 23 



a 



12 a \ 3 



a 



> 0 etc. 



22 



^13 ^23 a 33 



For A to be nonnegative definite, all minors whose diagonal entries are dia- 
gonal entries of A must be nonnegative. That is, for a 3 x 3 matrix A, 
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a xl a l3 a 22 #23 

a \3 a 33 a 23 a 33 

a \\ a \l a \3 
@12 ^22 ^2 3 

#13 ^23 #33 

A symmetric ^4 is positive definite if and only if its eigenvalues are posi- 
tive, and nonnegative definite if and only if its eigenvalues are nonnegative. 

If D is an n X m matrix, then A = DD' is nonnegative definite, and posi- 
tive definite if and only if D has rank n. An easy way to see this is to define 
a vector y by y = D'x. Then x’ Ax = xDD’x = y'y = ^yf > 0. The 
inequality becomes an equality if and only if y = 0 or D'x = 0, which is 
impossible for nonzero x if D has rank n. 

If A is nonnegative definite, there exists a matrix B that is a symmetric 
square root of A ; it is also nonnegative definite. It has the property that 

B 2 = A 

and is often denoted by A x/1 . If A is positive definite, so is A u 2 , and ,4 1/2 is 

then unique. 

If A and B are nonnegative definite, so is A + B , and if one is positive 
definite, so is A + B. If A is nonnegative definite and n x n, and B is m X n, 
then BAB' is nonnegative definite. 

If A is a symmetric matrix and A m&x is the maximum eigenvalue of A, 
then A max J — A is nonnegative definite. 

20. Norms of vectors and matrices. The norm of a vector x, written 
|| x ||, is a measure of the size or length of x. There is no unique definition, 
but the following postulates must be satisfied. 

1. || x || > 0 for all x with equality if and only if x = 0. 

2. ||ax|| = |a|||x|| for any scalar and for all x. 

3. || x + y\\ < || x || + || y || for all x and y. 

If x = (Xj, x 2 , . . . , xj, three common norms are 

11*11 = [l>< 2 ] ,/2 > ||*|| = max I*, | and ||*|| = JJ |x,|. 

i=l i i= 1 

The Schwartz inequality states that \x'y\ < ||x|| || y || for arbitrary 
x and y, with equality if and only if x = Ay for some scalar A. 

The norm of an m X n matrix A is defined in terms of an associated 
vector norm by 

||T|| — max ||j 4 x||. 

u*ii=i 

The particular vector norm used must be settled to fix the matrix norm. 



^ 1 1 > ^ 22 ’ ^33 — ^ 



a \\ a i2 
a \2 a 22 
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Corresponding to the three vector norms listed, the matrix norms become, 
respectively, [A max 04Vl)] 1/2 , max (2 1 |) and max (2 |tf j7 |). Important prop- 



erties of matrix norms are 



;= l 



i= 1 



!M.x||<|M||||x||, |M + 5||<|M|| + ||5[|, 



and 



\\AB\\ < IMH ||5||. 



21. Linear matrix equations. If A, B, and C are known matrices, of 
dimension n X n, m X w, and n X w, respectively, we can form the follow- 
ing equation for an unknown n X m matrix X: 

AX+XB + C^ 0. 

This equation is merely a condensed way of writing a set of mn simultaneous 
equations for the entries of X . It is solvable to yield a unique X if and only 
if XfA) + Xj{B) ^ 0 for any i and j — i.e., the sum of any eigenvalue of A 
and any eigenvalue of B is nonzero. 

If C is positive definite and A — B\ the lemma of Lyapunov states that 
X is positive definite symmetric if and only if all eigenvalues of B have nega- 
tive real parts. If C = DD' for some D , and A = B\ the lemma of Lyapunov 
states that X is nonnegative definite symmetric, and nonsingular if and only 
if [A, D] is completely controllable. 



22. Common differential equations involving matrices. The equation 

jXO) = A(t)x(t) x(t a ) = x 0 

commonly occurs in system theory. Here, A is n X w, and x is an n vector. 
If A is constant, the solution is 

x{t) = exp [A(J — f 0 )]x 0 . 

If A is not constant, the solution is expressible in terms of the the solution of 

- A(t)X(t) X(t 0 ) = /, 

where now X is an n x n matrix. The solution of this equation cannot nor- 
mally be computed analytically, but is denoted by the transition matrix 
r 0 ), which has the properties 

®('o, ? o) = I <&(* 2 > to) = ®{t 2 , t 0 ) 

and 

t 0 )$(t 0 , t) = /. 

The vector differential equation has solution 

x(t) = <£>(/, t 0 )x 0 . 
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The solution of 

= A(t)x(t) + B(t)u(t) x{t 0 ) ^ x 0 

where u(t) is a forcing term is 

x{t) = 3>(/, / 0 )x a + f 0(/, z)B(z)u(z) dz. 

J h 

The matrix differential equation 

d -* = AX+ XB + C(t) X(t a )=--X 0 

also occurs commonly. With A and B constant, the solution of this equation 
may be written as 

X{t) = exp [A{t — rJlA'o exp [B{t — /„)] 

+ f exp [A{t — z)]C(z) exp [ B(t — r)] dz. 

J to 

A similar result holds when A and B are not constant. 



23. Several manipulative devices. Let f(A) be a function of A such that 




oo 



2 a,A‘ 

— n 



(a i is constant). 



[In other words, /(z), where z is a scalar, is analytic.] Then, 



T~ l f(A)T - f(T~ l AT). 

This identity suggests one technique for computing f(A), if A is diagonaliz- 
able. Choose T so that T~ l AT is diagonal. Then f(T~ l AT) is readily com- 
puted, and f(A) is given by T AT)T~ l . It also follows from this identity 
that the eigenvalues of f(A) are f(Xi) where are eigenvalues of A\ the 
eigenvectors of A and j\A) are the same. 

For n vectors x and y, and A any n x n matrix, the following trivial 
identity is often useful: 

x Ay = y'A'x. 

If A is n x m, B is m X /?, I m denotes the m X m unit matrix, and I n 
the n x n unit matrix, then 

\I H +AB\ = \I m + BA\. 

If A is a column vector a and B a row vector b\ then this implies 

[ / + ab’ |=1+ b'a. 

Next, if A is nonsingular and a matrix function of time, then 






(Th is follows by differentiating A A 1 = 7.) 
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If F is n x n, G and K are n x r, the following identity holds: 

[/ + K'(sl — FpG ]' 1 = I — K\sl — F + GKT'G. 

Finally, if P is an n x n symmetric matrix, we note the value of 
grad (x'Px), often written just (i dldx)(x'Px ), where the use of the partial deriva- 
tive occurs since P may depend on another variable, such as time. As may be 
easily checked by writing each side in full, 

j-(x'Px) = 2 Px. 
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APPENDIX 



BRIEF REVIEW OF SEVERAL 

MAJOR RESULTS OF 
LINEAR SYSTEM THEORY 



This appendix provides a summary of several facts of linear system 
theory. A basic familiarity is, however, assumed. Source material may be 
found in, e.g., [1] through [3], whereas [4] deals with a special topic — the 
determination of state-space equations, given a transfer function matrix. 

1. Passage from state-space equations to transfer function matrix. In 

system theory, the equations 

x = Fx + Gu 
y = H'x 

frequently occur. The Laplace transform may be applied in the same manner 
as to scalar equations to yield 

sX(s) = FX(s) + X(0) + GU(s) 

Y(s ) = H'X(s) 

whence 

Y(s) = H'(sl - F)-'GU(s) 

with Z(0) = 0. The transfer function matrix relating U(s) to Y(s) is 
H'(sI-F)~'G. 
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2, Conditions for complete controllability and observability. A pair of 
constant matrices [F, G] with F n x n and G n x r is termed completely 
controllable if the following equivalent conditions hold; 

1. Rank [G FG - * • F n ~ x G\ - n. 

2. We Ft G — 0 for all t implies w = 0. 

3. Jq e Vt GG'e rt dt is positive definite for all T > 0. 

4. There exists an n x r matrix /C such that the eigenvalues of F + GK' 
can take on arbitrary prescribed values. 

5. Given the system x = Fx + Gu , arbitrary states x 0 , x 15 and arbitrary 
times / 0 , with / 0 < there exists a control taking the system from 
state x 0 at / 0 to state Xj at t t . [In contrast to (1) through (4), this is 
also valid for time-varying F and G if t 0 — t Q (/ 1# x 0 , Xj).] 

A pair of matrices [F, //] with F n x n and H n x r is termed completely 
observable if [F\ H] is completely controllable. 

3. Passage from transfer function matrix to state space equations — the 
Ho algorithm. The determination of state-space equations corresponding 
to a transfer function, as distinct from transfer function matrix, is straight- 
forward. Given a transfer function 

W(s\ — 1 + b„_ ] s n ~ 2 + • - * + b { 

K } s n + a„s n ~ l + *•• + a x 

state-space equations x — Fx + gu, y = h'x yield the same transfer function 
relating U(s) to Y(s) if 
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These formulas are valid irrespective of whether the numerator and 
denominator of W(s) have common factors. The first pair of F and g is com- 
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pletely controllable, and there exists a coordinate basis transformation taking 
any other set of F, g and h which are completely controllable to the prescribed 
form. The second pair of F and h is completely observable, and there exists 
a coordinate basis transformation taking any other completely observable 
set to the prescribed form. 

If the numerator and denominator of W(s) have no common factor, both 
sets are simultaneously completely controllable and observable. 

When W(s) is a matrix — say, p x m — the algorithm due to Ho [4] pro- 
vides a convenient route to determining matrices F, G , and H. First, W(s) 
is assumed to be zero at s = oo. It is then expanded as 




where the A { are termed Markov matrices . Then the A t are arranged to form 
Hankel matrices H N as follows: 
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The next step requires the checking of the ranks of H N for different AT, 
to determine the first integer r such that rank H r = rank H r¥ x = rank H r+2 
= * • * - If IV(s) is rational, there always exists such an r. Then nonsingular 
matrices P and Q are found so that 



PHrQ 



In 0 ' 

o_ 



where n = rank H r . The following matrices “realize” lV(s), in the sense that 
lV(s) = H'(sl — F)~ X G: 



where 



G = n x m top left corner of PH f 
H f = p x n top left corner of H r Q 
F — n x n top left corner of P(oH r )Q 




Moreover, [F, G] is completely controllable and [F, H] is completely observ- 
able. 
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4. Minimality. If a transfer function matrix W(s) is related to a matrix 
triple F, G, H by 

W(s) - H'(sl — F)~ X G, 

then Fhas minimal dimension if and only if [F, G ] is completely controllable 
and [F, H] is completely observable. The triple F, <7, // is termed a minimal 
realization of W(s). Given two minimal realizations of W{s) — call them F,, 
G x , H 1 and F 2 , G 2 , H 2 — there always exists a nonsingular T such that 

FFjF 1 = F 2 TG { - G 2 (T' 1 ) , H l = H 2 . 
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